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FOREWORD 


The  Army  has  a  continuing  research  need  to  investigate  the  effects  of  and 
tradeoffs  among  alternative  devices  and  strategies  for  training  gunnery  skills. 
This  report  is  one  in  a  series  that  concerns  the  development  of  research  tools 
for  supporting  such  research.  Previous  work  in  this  series  has  described  the 
development  and  application  of  methods  for  determining  valid  threat  engagement 
scenarios  for  training  gunnery  skills,  and  methods  for  identifying  gunnery 
training  objectives  that  may  be  trained  in  those  engagement  scenarios.  In 
other  words,  the  previous  work  has  established  what  is  to  be  trained.  In  con¬ 
trast,  the  focus  of  this  report  is  on  how  the  objectives  should  be  trained. 

The  specific  purpose  of  this  report  is~To  describe  and  apply  methods  that  ad¬ 
dress  four  central  problems  that  impact  the  design  of  gunnery  training:  the 
organization  of  training  objectives  into  units  of  instruction,  the  sequence  of 
training  both  within  and  among  units  of  instruction,  the  selection  of  an  ap¬ 
propriate  training  device  or  medium  for  each  unit,  and  the  allocation  of  train¬ 
ing  time  to  each  unit/device  combination. 

The  methods  described  in  this  report  represent  significant  contributions 
to  the  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
Exploratory  Development  research  program.  The  audience  for  this  report  in¬ 
cludes  scientists  who  perform  research  related  to  the  evaluation  of  training 
strategies  and  devices.  However,  these  same  methods  should  also  be  of  inter¬ 
est  to  those  who  develop  actual  military  training.  This  research  is  part  of 
the  ARI  task  entitled  "Application  of  Technology  to  Meet  Armor  Skills  Training 
Needs."  It  is  performed  under  the  auspices  of  ARI's  Armor  Research  and  Devel¬ 
opment  Activity  at  Fort  Knox.  The  proponent  for  the  research  is  the  Deputy 
Chief  of  Staff,  Training,  U.S.  Army  Training  and  Doctrine  Command. 


DESIGNING  A  GUNNERY  TRAINING  STRATEGY 


EXECUTIVE  SUMMARY 


Requirement: 

The  Army  has  a  continuing  research  need  to  investigate  the  effects  of  and 
tradeoffs  among  alternative  devices  and  strategies  for  training  gunnery  skills. 
This  report  is  one  in  a  series  that  concerns  the  development  of  research  tools 
for  supporting  such  research.  Previous  research  in  this  series  defined  the 
context  and  the  content  of  crew-  and  platoon-level  gunnery  training.  The  pur¬ 
pose  of  the  present  research  was  to  develop  and  apply  strategies  for  training 
gunnery  skills. 


Procedure: 

A  training  strategy  was  defined  as  a  method  for  systematically  config¬ 
uring  training  events.  To  apply  the  concept  of  training  strategies  to  the 
design  of  armor  gunnery  training,  strategies  were  further  distinguished  by 
four  general  problems  that  they  address  (a)  the  derivation  of  an  appropriate 
structure  that  identifies  units  of  instruction,  (b)  the  sequencing  of  training 
both  within  and  among  units  of  instruction,  (c)  the  selection  of  an  appropri¬ 
ate  training  device  or  medium  for  each  unit,  and  (d)  the  allocation  of  training 
time  to  each  unit/device  combination.  The  training  strategy  methods  were  first 
discussed  in  terms  of  their  theoretical  rationale  and  their  usefulness  for  gun¬ 
nery  training.  The  methods  were  then  tested  by  applying  them  to  two  prototypi¬ 
cal  problems  in  gunnery  training.  The  first  problem  addressed  the  use  of  two 
different  devices  for  training  similar  basic  gunnery  skills,  whereas  the  second 
problem  concerned  the  use  of  multiple  devices  to  train  dissimilar  skills  in 
platoon  gunnery. 


Findings: 

The  application  of  methods  for  designing  a  training  strategy  demonstrated, 
in  general,  that  the  various  methods  could  be  applied  to  dissimilar  gunnery 
training  problems  with  sensible  results.  Only  one  of  the  methods  failed  to 
apply  to  both  problems.  The  methods  for  allocation  of  training  time  were  only 
applicable  to  the  simpler  gunnery  training  problem  where  only  two  training  de¬ 
vices  were  considered;  the  methods  for  allocating  training  time  were  not  able 
to  handle  the  more  difficult  multi-device  tactical  training  problem.  Some  of 
the  other  problems  were  also  noted  relating  to  the  fact  that  some  of  the  meth¬ 
ods  are  not  as  fully  developed  as  others;  that  is,  they  were  stated  as  heuris¬ 
tic  guidelines  rather  than  as  fully  developed  algorithms. 


Utilization  of  Findings: 

The  methods  discussed  in  the  present  report  are  primarily  intended  for 
use  by  scientists  who  perform  research  related  to  the  evaluation  of  training 
strategies  and  devices.  However,  these  same  methods  should  also  be  of  inter¬ 
est  to  those  who  develop  actual  military  training.  Furthermore,  although  the 
methods  of  the  present  report  were  tested  on  armor  skills,  the  training  strat¬ 
egies  should  nevertheless  apply  to  training  in  any  combat  arm. 
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DESIGNING  A  GUNNERY  TRAINING  STRATEGY 


Introduction 

The  Army  has  a  continuing  research  need  to  investigate  the  effects  of 
and  tradeoffs  among  alternative  devices  and  strategies  for  training  gunnery 
skills.  The  present  report  is  one  in  a  series  that  concerns  the  development 
of  research  methods  and  tools  for  supporting  this  research.  The  purpose  of 
the  present  report  is  to  derive  training  strategies  that  are  designed  to 
enhance  tank  gunnery  training,  and  to  demonstrate  their  application  on  some 
representative  problems  in  gunnery  training.  These  strategies  serve  both 
primary  and  secondary  functions.  The  primary  function  of  these  strategies  is 
to  provide  guidance  for  development  of  experimental  training  procedures  and 
materials  required  to  examine  other  research  issues,  for  example,  the 
derivation  of  skill  acquisition  and  transfer  functions.  This  guidance  is 
based  on  the  current  learning  theory  and  findings  from  the  literature.  The 
strategies  themselves  range  from  well  known  methods  that  are  supported  by  a 
wealth  of  data  to  theoretically  plausible  methods  that  are  supported  by 
limited  findings.  Strategies  on  the  latter  end  of  the  spectrum  can  provide 
the  focus  for  further  research.  Thus,  the  secondary  function  of  training 
strategies  is  to  generate  testable  hypotheses  about  the  conduct  of  gunnery 
training. 


Previous  Research  in  This  Series 


Previous  work  in  the  series  on  gunnery  research  has  described  the 
development  and  application  of  methods  for  determining  valid  threat  engagement 
scenarios  for  training  gunnery  skills  (R.  Campbell  &  C.  Campbell,  1990;  Doyle, 
1990),  and  methods  for  identifying  gunnery  training  objectives  that  may  be 
trained  in  those  engagement  scenarios.  These  threat  scenarios  described  key 
engagements  that  are  likely  to  occur  within  the  context  of  the  execution  of 
various  Red  (threat)  and  Blue  (friendly)  mission  combinations.  The  detailed 
scenario  information  included  the  specification  of  the  number,  type,  and  range 
of  likely  threat  vehicles.  In  summary,  this  initial  research  established  the 
context  for  gunnery  training  and  performance  measurement. 

A  subsequent  report  in  the  series  (Morrison,  Meade,  &  Campbell,  1990) 
was  concerned  with  the  development  and  application  of  methods  for  determining 
training  objectives  that  are  appropriate  to  these  scenarios.  Objectives  were 
defined  as  subtasks  performed  at  crew  or  platoon  level.  Cluster  analyses  of 
the  crew-level  objectives  indicated  eight  distinct  categories  of  subtasks,  the 
largest  of  which  were  continuous  control,  psychomotor  subtasks  related  to 
manipulation  of  control  handles  in  both  machine  gun  and  main  gun  engagements. 
Other  categories  of  subtasks  related  to  topics  such  as  (a)  the  control  of 
gunnery  engagements  by  the  tank  commander,  (b)  target  acquisition, 

(c)  immediate  action  procedures,  (d)  switch  setting  procedures,  (e)  fire 
commands  and  reporting  duties,  (f)  decisions  and  actions  related  to  operation 
in  degraded  modes,  and  (g)  tank  maneuvering  skills.  At  the  platoon  level  of 
analysis,  two  types  of  subtasks  were  identified:  (a)  individual  platoon 
leadership  subtasks,  cognitive/verbal  actions  related  to  platoon  coordination 
both  internal  and  external;  and  (b)  platoon  collective  subtasks,  actions 
characterized  by  the  predominance  of  interactive  and  coord i native  skills.  In 


1 


short,  this  analysis  indicated  that  the  contents  of  gunnery  training  consist 
of  an  extensive  and  heterogeneous  set  of  training  objectives. 

Training  Strategies 

Having  established  the  context  for  and  the  contents  of  a  gunnery 
training  program,  the  next  step  in  training  development  was  to  specify  how 
training  should  be  configured.  We  use  the  term  "training  strategy"  to 
differentiate  the  consideration  of  this  aspect  of  training  development  apart 
from  the  specification  of  training  context  and  content.  There  are  a  wide 
variety  of  training  strategies  that  could  be  applied  to  specific  training 
objectives.  For  example,  Morrison  and  Walker  (1989)  applied  mental  practice 
techniques  to  learning  the  procedural  and  psychomotor  skills  related  to 
gunnery.  There  are  far  too  many  of  these  types  of  specific  strategies  to 
comprehensively  review  in  one  report.  Furthermore,  these  specific  strategies 
are  limited  by  their  restricted  range  of  applications. 

In  contrast  to  the  numerous  specific  strategies  mentioned  in  the 
previous  paragraph,  there  is  a  smaller  number  of  more  broadly  defined 
strategies  which  are  designed  to  apply  across  dissimilar  types  of  objectives. 
These  latter  types  of  training  strategies  are  defined  as  instructional  methods 
that  address  some  general  problems  of  training  design.  One  problem  in 
providing  a  precise  definition  of  a  "training  strategy"  is  that  researchers 
have  used  the  term  to  refer  to  methods  that  apply  to  one  or  maybe  two  of  these 
general  problems  to  the  exclusion  of  methods  that  apply  to  the  other  problems. 
To  develop  a  definition  of  a  "training  strategy"  that  is  relevant  to  gunnery 
training  needs,  it  useful  to  first  identify  the  types  of  general  problems  that 
are  particularly  troublesome  for  gunnery  training.  These  problems  serve  to 
define  the  components  of  a  comprehensive  strategy  for  training  gunnery  skills. 

The  training  development  literature  identifies  strategies  related  to  two 
different  but  related  problems:  determining  the  appropriate  structure  and 
determining  the  appropriate  sequence  of  training  (Reigeluth  &  Curtis,  1987). 
The  structure  of  training  refers  to  the  organization  of  individual  objectives 
into  instructional  units  that  serve  as  the  targets  of  training  design.  The 
sequence  of  training  refers  to  a  prescription  of  the  order  in  which  the 
objectives  should  be  addressed,  both  within  anc  among  instructional  units. 

The  two  problems  are  related  because  they  both  relate  to  the  configuration  of 
training.  For  a  small  number  of  homogeneous  objectives,  the  appropriate 
conf iguration  may  be  obvious  to  the  training  developer.  However,  the  domain 
of  gunnery  training  objectives  is  neither  small  nor  homogeneous.  Given  the 
nature  of  the  domain  of  gunnery  training  objectives,  systematic  strategies  are 
required  to  structure  and  sequence  training. 

Training  managers  and  developers  in  the  Armor  community  commonly  use 
"training  strategy"  to  refer  to  methods  for  addressing  the  problem  of 
selecting  training  devices  or  media.  The  problem  of  selecting  training 
devices  is  particularly  acute  for  armor  training  given  the  number  of  devices 
that  address  gunnery  skills.  Hoffman  and  Morrison  (1988)  identified  four 
computer-based  devices  that  could  be  used  to  train  gunnery  skills.  Analysis 
of  device  capabilities  indicated  substantial  overlap  in  the  training  functions 
of  devices.  The  decision  that  the  training  developer  must  make  is  to  choose 
the  device  that  is  appropriate  for  a  particular  skill  and  that  also  conforms 
to  the  structure  and  sequence  of  the  course.  To  the  computer-based  devices 
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addressed  by  Hoffman  and  Morrison,  the  present  report  adds  a  variety  of  new 
tank-appended  technologies  for  training  crew-  and  platoon-level  skills, 
thereby  complicating  the  device  selection  problem.  Thus,  device  selection 
methods  are  key  components  to  a  gunnery  training  strategy. 

Researchers  who  specialize  in  the  effects  of  training  devices  (e . g . , 
Roscoe  &  Williges,  1980)  have  used  training  strategies  to  refer  to  methods 
that  apply  to  yet  another  problem  in  training  design:  the  allocation  of 
training  time  between  a  training  device  and  the  actual  equipment.  More 
generally  stated,  this  type  of  training  strategy  refers  to  allocation  of 
training  time  among  all  training  media,  including  the  operational  equipment. 
The  purpose  of  this  training  strategy  is  to  allocate  training  time  such  that 
performance  is  maximized  or  that  training  costs  are  minimized.  This 
particular  training  strategy  is  important  for  gunnery  training  because  of  the 
device  alternatives  differ  in  terms  of  their  effectiveness  to  train  certain 
objectives,  and  in  terms  of  the  costs  associated  with  delivering  that 
training.  Thus,  the  allocation  strategy  must  consider  both  factors  (training 
effectiveness  and  costs)  in  determining  appropriate  training  times  for 
devices. 

In  summary,  training  strategies  can  be  discussed  in  the  context  of  four 
general  problems  that  seriously  impact  the  design  of  gunnery  training: 

(a)  the  derivation  of  an  appropriate  structure  that  identifies  units  of 
instruction,  (b)  the  sequencing  of  training  both  within  and  among  units  of 
instruction,  (c)  the  selection  of  an  appropriate  training  device  or  medium  for 
each  unit,  and  (d)  the  allocation  of  training  time  to  each  unit/device 
combination.  Any  comprehensive  gunnery  training  strategy  must  address  all  of 
these  fundamental  prob1ems. 

Specific  Objectives  and  Report  Organization 

The  purpose  of  the  present  report  is  to  describe  and  to  apply  training 
strategies  that  address  the  four  aforementioned  problems.  The  report  is 
divided  into  two  major  sections.  The  first  section  describes  the  methods  for 
designing  training  strategies  by  presenting  their  theoretical  rationale  and 
evaluating  their  usefulness  for  gunnery  training;  thus,  it  is  referred  to  as 
the  "methods"  section.  The  second  section  presents  the  same  methods  in  a  more 
applied  context  by  discussing  them  in  the  context  of  two  prototypical  problems 
in  gunnery  training;  thus,  the  second  section  is  referred  to  as  the 
"applications"  section. 

Methods  for  Designing  a  Training  Strategy 

Many  methods  related  to  the  design  of  training  strategies  are  compiled 
in  the  Interservice  Procedures  for  Instructional  Systems  Development  (ISD) 
(Branson  et  al . ,  1975).  While  some  of  the  ISD  procedures  serve  as  useful 
points  of  departure,  the  present  review  concentrates  on  methods  that  have 
emerged  during  the  15  years  since  the  publication  of  that  influential 
document.  It  should  be  noted  at  the  outset  that  the  methods  described  in  the 
present  report  were  not  derived  from  a  single  theoretical  point  of  view. 
Rather,  they  comprise  an  eclectic  group  of  methods  that  were  selected  because 
they  were  logical,  practicable,  and/or  supported  by  empirical  findings. 
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This  present  section  presents  a  wide  array  of  methods  for  designing 
training  strategies--some  of  which  are  well-known,  others  have  been  developed 
specifically  for  the  present  project.  Despite  the  diversity  of  methods,  each 
can  be  described  as  being  either  algorithmic  or  heuristic.  Apter  (1970) 
defines  an  algorithm  as  an  “.  .  .  unambiguously  stated  'machine-like' 
procedure  which,  if  followed  literally,  will  inevitably  achieve  the  intended 
result"  (p.  40).  Arithmetic  procedures  (e . g. ,  adding,  subtracting, 
multiplying,  or  dividing)  or  computer  programs  are  examples  of  algorithmic 
methods.  Simple  algorithms  can  be  described  as  a  series  of  sequential  steps, 
while  more  complex  branching  algorithms  are  better  described  in  terms  of 
flowcharts.  An  example  of  an  algorithmic  method  for  specifying  training 
strategies  is  the  set  of  learning  guidelines  presented  in  the  Interservice 
Procedures  for  Instructional  Systems  Development  (Branson  et  al . ,  1975). 

These  guidelines  were  presented  as  a  set  of  algorithms  for  prescribing  the 
learning  activities  that  should  occur  during  training.  The  particular 
algorithm  used  depends  on  the  category  of  information  processing  that  is  most 
characteristic  of  the  task(s)  being  trained,  for  example,  rule  learning/using, 
classifying/recognizing,  identifying  symbols,  and  performing  gross  motor 
skills. 

In  contrast  to  an  algorithm,  a  heuristic  is  "...  a  set  of  rules  which, 
if  followed,  may  achieve  a  solution  but  cannot  guarantee  to  do  so"  (Apter, 
1970,  p.  83).  The  instructional  sequencing  procedure  described  by  Branson  et 
al.  (1975)  is  an  example  of  a  heuristic  method  that  is  embodied  as  a  loosely 
defined  set  of  rules.  For  instance,  one  of  their  rules  (critical  sequence) 
states  that  learning  objectives  should  be  ordered  from  most  to  least 
important.  Another  (simple  to  complex)  states  that  objectives  should  be 
ordered  in  terms  of  increasing  complexity.  To  see  how  these  guidelines  apply, 
consider  two  gunnery  subtasks  identified  by  Morrison  et  al.  (1990):  lase  to 
target  and  fire  at  target.  Firing  at  the  target  is  clearly  the  more  critical 
subtask  of  the  two  in  terms  of  accomplishing  the  overall  task  goal,  that  is, 
to  hit  the  target.  On  the  other  hand,  lasing  is  more  complex  because  the 
gunner  is  not  only  required  to  hit  the  target  (with  a  laser  beam,  not  a 
round),  but  the  gunner  must  also  evaluate  the  laser  return  quickly  to  decide 
whether  to  lase  again  or  go  ahead  and  fire  at  the  target.  Because  these  two 
rules  lead  to  contradictory  conclusions,  the  solution  (i.e.,  the  instructional 
sequence  for  these  two  objectives)  cannot  be  determined  from  these  two 
heuristics  alone. 

Superficially,  algorithmic  methods  appear  more  rigorous  than  heuristic 
methods.  Indeed,  one  could  argue  that  algorithmic  formats  should  be  the 
eventual  goal  in  the  methodological  development  of  training  strategies.  Given 
the  present  state  of  knowledge  about  training,  however,  heuristic  guidelines 
are  often  the  only  realistic  type  of  prescription  that  may  be  provided  to 
training  designers.  Furthermore,  algorithms  can  be  crippled  by  unrealistic 
simplifying  assumptions  that  reduce  their  range  of  applicability.  For 
instance,  the  training  prescription  algorithms  presented  by  Branson  et  al. 
(1975)  apply  to  only  11  categories  of  tasks.  Thus,  this  algorithmic  approach 
tacitly  assumes  that  tasks  can  be  reliably  sorted  into  mutually  exclusive 
learning  categories--an  assumption  that  has  been  challenged  on  a  number  of 
fronts  (e.g.,  Boldovici,  Harris,  Osborn,  &  Heinecke,  1977;  Vineberg  &  Joyner, 
1980).  Another  drawback  to  mathematical  algorithms  is  that  necessary  input 
data  may  not  be  available  and  may  be  difficult  to  generate.  For  instance,  the 
methods  for  allocating  training  time  are  based  on  determining  transfer 
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functions.  As  is  discussed  later  in  this  methods  section,  the  data  collection 
requirements  for  obtaining  the  transfer  data  are  not  trivial. 

The  following  descriptions  of  training  strategy  methods  are  organized 
according  to  the  four  general  problems  identified  in  the  Introduction: 

(a)  structure  training,  (b)  sequence  training,  (c)  select  media,  and 
(d)  allocate  training  time.  It  should  be  noted  that  the  four  groups  of 
methods  differ  with  respect  to  the  characteristics  discussed  above.  The 
methods  associated  with  the  first  two  problems  (structuring  and  sequencing 
training)  are  heuristic  in  nature;  that  is,  the  methods  are  basically 
guidelines  to  training  design  rather  than  explicit  procedures.  The  appendix 
presents  an  attempt  to  develop  the  methods  further  by  formalizing  the  use  of 
those  guidelines.  Although  the  methods  for  structuring  and  sequencing 
training  are  not  well  developed,  their  execution  requires  only  knowledge  of 
training  methods  and  the  objectives  themselves.  In  contrast,  the  methods 
related  to  the  latter  two  problems  (selecting  media  and  allocating  training 
time)  are  based  on  computational  algorithms  that  are  quite  well  developed.  On 
the  negative  side,  the  data  for  executing  these  algorithms  are  more  difficult 
to  obtain.  After  the  description  of  each  training  strategy  method,  a  brief 
evaluation  of  the  method  is  offered.  In  some  cases,  the  evaluation  summarizes 
the  research  literature  for  and  against  the  methods.  In  other  cases, 
empirical  research  is  proposed  to  resolve  methodological  problems. 

Structure  Training 

There  are  two  purposes  that  are  commonly  cited  for  structuring  or 
organizing  training.  The  ISD  documentation  (Branson  et  al.,  1975)  points  out 
that  one  purpose  of  structuring  training  is  to  facilitate  the  development  and 
implementation  of  training  by  creating  independent  "modules"  of  instruction. 
This  first  purpose  implies  that  the  structure  of  training  depends  on  practical 
considerations  such  as  administrative  convenience  and  media  availability. 

While  it  is  clear  that  these  factors  are  important  to  the  structuring  process, 
it  is  also  clear  that  these  factors  are  specific  to  particular  training 
applications.  The  second,  and  perhaps  more  fundamental,  purpose  for 
structuring  training  is  to  promote  the  acquisition,  retention,  and  transfer  of 
gunnery  skills.  The  following  review  focuses  on  factors  that  relate  to  this 
second  purpose  for  structuring. 

The  process  of  structuring  training  requires  the  identification  of 
relationships  that  exist  among  training  objectives.  The  following  sections 
discuss  two  types  of  possible  relations  that  may  exist  among  objectives  and 
that  affect  skill  acquisition.  The  first  section  provides  a  discussion  of  the 
concept  of  dependent  and  supportive  relations  among  objectives—traditional 
concepts  in  the  training  development  literature.  These  types  of  relations 
correspond  to  the  concepts  of  learning  prerequisites  and  transfer  of  training, 
respectively.  In  addition  to  implications  for  structuring  training,  these 
concepts  also  have  implications  for  the  order  of  training,  and  are 
consequently  discussed  in  the  context  of  the  next  problem  (sequencing 
training).  The  second  and  third  sections  under  the  present  problem  focus  on 
some  newer  concepts  that  have  evolved  from  recent  research  on  part-task 
training:  the  extent  to  which  sequentially  related  subtasks  address  common 
task  goals  and  the  extent  to  which  time-shared  subtasks  share  similar 
attentional  resources. 
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Dependent  and  Supportive  Relations 

In  the  ISD  procedures,  Branson  et  al.  (1975)  distinguished  between 
dependent  and  supportive  relationships.  A  dependent  relationship  is  defined 
wherever  the  learning  of  one  objective  is  necessary  in  order  to  learn  another. 
For  instance,  Morrison  and  Hoffman  (1988)  identified  the  skill  of  tracking 
armor  gunnery  targets  as  dependent  on  the  more  basic  skill  of  knowing  how  to 
manipulate  the  gunner's  control  handle.  In  contrast,  a  supportive 
relationship  is  defined  wherever  the  learning  of  one  objective  is  helpful  to 
(but  not  required  for)  learning  the  other.  In  other  words,  skills  learned  in 
the  first  objective  transfer  to  the  second.  This  latter  sort  of  transfer 
relationship  occurs  because  of  similarities  between  objectives  in  their 
conditions  and/or  behaviors.  For  instance,  the  skills  related  to  engaging 
main  gun  targets  and  engaging  coax  targets  are  not  dependent  upon  one  another. 
Nevertheless,  the  two  types  of  skills  are  sufficiently  similar  to  expect 
positive  transfer  from  the  learning  of  one  type  of  engagement  to  the  learning 
of  the  other. 

Although  dependent  and  supportive  relationships  may  be  conceptually 
distinguished  from  one  another,  their  implications  for  a  training  strategy  are 
similar:  To  the  extent  possible,  related  objectives  should  be  kept  in  close 
proximity  to  one  another.  Dependency  related  objectives  should  be  kept 
together  to  prevent  the  prerequisite  skills  of  the  dependent  skill  from  being 
forgotten  over  time  or  through  interference  with  other  learning  before 
learning  the  superordinate  objective.  Similarly,  transfer  between  supportive 
objectives  is  maximized  when  objectives  are  located  in  close  temporal 
proximity  to  one  another  such  that  forgetting  and  interference  effects  are 
minimized. 

Methods.  Dependent  relationships  are  usually  identified  through 
rational,  heuristic  methods  called  hierarchical  learning  analysis  of  task 
prerequisites  (e.g.,  Gagn§,  1967;  Resnick,  1976).  Hierarchical  analysis 
starts  by  examining  the  terminal  learning  objective  to  identify  prerequisite 
objectives,  that  is,  those  that  are  necessary  for  a  student  to  learn  before 
learning  the  terminal  objective.  Each  prerequisite  skill  is  then  examined  to 
determine  whether  or  not  lower  level  prerequisites  need  to  be  identified. 

This  analysis  stops  when  it  reaches  prerequisite  skills  that  may  be  assumed  to 
be  possessed  by  all  members  of  the  to-be-trained  populations.  The  results  of 
the  analysis  are  usually  summarized  by  an  inverted  tree  structure  with  the 
terminal  training  objective  placed  at  the  top  of  the  tree  with  prerequisite 
relations  represented  as  branches  connecting  prerequisites  to  the 
superordinate  objective  that  they  support. 

Supportive  relationships  are  usually  identified  through  an  analysis  of 
the  similarity  of  training  objectives  in  terms  of  their  stimuli  and/or 
responses.  Predictions  of  positive  and  negative  transfer  were  summarized  by 
Osgood's  (1949)  transfer  surface,  which  was  originally  based  on  results  from 
verbal  and  simple  motor  learning  experiments.  Holding  (1976)  modified  this 
surface  based  on  more  recent  results  from  the  skill  learning  literature. 

Though  different  in  their  particulars,  two  key  "principles"  for  predicting 


hierarchical  relationships  are  further  examined  in  the  next  section  in 
connection  with  the  sequencing  of  training. 
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transfer  can  be  derived  from  either  surface:  (a)  positive  transfer  will  occur 
between  two  tasks  when  their  stimuli  are  similar  and  when  their  responses  are 
similar;  and  (b)  negative  transfer  will  occur  when  the  responses  to  the 
similar  stimuli  are  different.  Similar  transfer  principles  are  implicit  in 
models  that  the  Army  has  developed  to  predict  the  effectiveness  of  training 
devices.  (For  a  review  of  these  models,  see  Tufano  &  Evans,  1982). 

Evaluation.  Although  there  is  scant  evidence  that  organizing  training 
with  respect  to  dependent  relationships  actually  promotes  skill  acquisition 
(e.g.,  White,  1973),  this  particular  strategy  is  congruent  with  standard 
educational  practice.  Hoffman  and  Morrison  (1988)  applied  hierarchical 
analysis  techniques  to  examine  gunnery  training  objectives.  They  found  that 
gunnery  does  not  have  an  elaborate  structure  of  prerequisites.  The  lack  of 
prerequisites  implies  that  the  number  of  dependent  relationships  in  gunnery  is 
probably  limited. 

With  respect  to  supportive  relationships,  there  is  substantial  evidence 
from  the  research  literature  substantiating  the  claim  that  amount  or  degree  of 
transfer  between  tasks  is  a  positive  function  of  their  similarity.  The 
problem  is  in  precisely  defining  similarity.  Osgood's  and  Holding's  transfer 
surfaces  define  similarity  on  the  basis  of  task  stimuli  and  responses.  It 
should  be  noted  that  the  generalizations  from  the  transfer  surface  are  largely 
based  on  findings  from  verbal  paired  associate  learning  where  stimuli  and 
response  are  well  differentiated.  In  real-world  psychomotor  tasks  such  as 
those  related  to  gunnery,  stimuli  and  responses  are  not  as  distinguishable. 
Target  tracking  in  gunnery  provides  a  prototypical  example.  Initially,  the 
response  in  tracking  is  to  manipulate  the  gunner's  control  handle.  The 
response  input,  however,  feeds  back  into  the  system  to  change  the  sight 
picture  (i.e.,  the  visual  stimuli)  so  that  stimuli  and  response  become  highly 
intertwined.  As  a  consequence,  analysis  by  transfer  surfaces  would  be 
difficult  to  apply  directly  to  many  armor  skills. 

Commonality  in  Subooals 

The  process  of  dividing  tasks  into  sequential  parts  for  training  has 
been  termed  segmentation  (Naylor,  1962).  Several  researchers  have  suggested 
that  segmentation  should  be  congruent  with  the  task  subgoal  structure 
(Hoffman,  Drucker,  Morrison,  &  Goldberg,  1983;  Knerr  et  a 1 . ,  1986;  Morrison, 
1984).  That  is,  the  "parts"  for  part  training  by  segmentation  should  be 
behaviors  related  by  the  fact  that  they  share  a  common  task  subgoal.  For 
instance,  Morrison  showed  that  the  sequential  behaviors  required  to  clear  a 
machine  gun  could  be  related  to  one  or  the  other  of  two  task  subgoals:  unload 
the  gun  and  return  the  bolt  to  the  forward  position.  According  to  a 
segmentation  strategy,  this  machine  gun  task  should  be  partitioned  for  part- 
task  training  in  accordance  with  those  two  goals. 

Methods.  Researchers  have  demonstrated  that  relationships  among  task 
behaviors  and  subgoals  can  be  identified  through  either  rational  methods 
(i.e.,  task  analytic)  or  an  empirical  analysis  of  performance.  Morrison  and 
Goldberg  (1982)  reviewed  the  cognitive  literature  to  reveal  guidelines  for 
rationally  parsing  tasks  into  goals  and  subgoals.  They  applied  these 
guidelines  to  the  analysis  of  several  armor  procedures.  In  a  follow-up  study, 
Morrison  (1984)  described  a  procedure  that  was  based  on  an  empirical  analysis 
of  recall  performance.  This  analysis  of  recall  is  based  on  Friendly's  (1979) 
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assumption  that  items  grouped  together  in  memory  tend  to  be  recalled  closely 
together.  Thus,  the  organization  of  memory  for  a  task  can  be  derived  from  the 
pattern  of  interresponse  times  in  free  recall  of  the  task  elements.  The  two 
approaches  to  determining  these  relationships  were  mutually  confirming  in  two 
senses:  (a)  the  rational  analysis  aided  in  interpreting  the  results  from  the 
analysis  of  free  recall,  and  (b)  the  proximity  analysis  of  free  recall 
confirmed  and  elaborated  on  the  rational  analysis  of  task  subgoals. 

Three  recent  studies  on  flight  tasks  (Bailey,  Hughes,  &  Jones,  1980; 
Westra,  1982;  Wightman,  1983)  have  demonstrated  that  part  training  by 
segmentation  produces  superior  performance  than  training  on  the  whole  task. 
These  studies  rationally  segmented  complex  flight  tasks  (30°  dive  bomb  for  the 
former,  carrier  landing  for  the  two  latter  studies)  into  sequential  segments. 
Part  training  using  a  recombination  procedure  called  "backward  chaining"  (see 
section  on  sequencing  training)  was  successful  in  producing  greater  learning 
than  equivalent  amounts  of  whole  task  learning. 

Although  these  findings  support  the  rule  to  group  behaviors  related  by 
common  subgoals,  the  demonstrated  difference  between  part  and  whole  training 
could  be  predicted  from  other  theoretical  points  of  view.  In  fact,  part 
training  of  complex  and  difficult  tasks  is  appealing  and  predictable  from  the 
layman's  point  of  view  (Adams,  1960).  A  more  direct  test  of  the  effects  of 
subgoal  commonality  would  not  compare  part  and  whole  training;  it  would 
instead  compare  groups  whose  training  is  either  organized  or  not  organized  in 
accordance  with  the  task  subgoal  structure.  Consider  a  task  having  three 
subgoals  (A,  B,  C)  with  four  behaviors  associated  with  the  first  subgoal  (Al, 
A2,  A3,  A4);  three,  with  the  second  (Bl,  B2,  B3);  and  two,  with  the  third  (Cl, 
C2).  According  to  the  subgoal  commonality  view,  training  should  be  organized 
in  three  units,  each  corresponding  to  the  three  subgoals.  In  contrast,  a 
suboptimal  organization  may  specify  that  objectives  into  three  units 
consisting  of  three  behaviors  each:  (Al,  A2,  A3),  (A4,  Bl,  B2),  (B3,  Cl,  C2). 
To  date,  no  comparison  of  alternative  part  task  procedures  has  been  performed. 

Evaluation.  In  summary,  this  review  indicated  that  there  are 
theoretical  (as  well  as  common-sense)  reasons  for  organizing  objectives  on  the 
basis  of  common  subgoals.  Furthermore,  to  the  extent  that  many  gunnery  tasks 
are  sequential  in  nature,  this  rule  would  appear  to  have  particular  relevance 
to  gunnery.  The  rational  and  empirical  methods  for  determining  memory 
organization  are  potentially  useful  for  determining  an  appropriate  structure 
for  gunnery  training,  although  there  is  a  lack  of  empirical  evidence 
indicating  that  this  structure  does,  in  fact,  facilitate  skill  acquisition. 

Commonality  in  Attentional  Resources 

In  contrast  to  sequential  tasks,  the  components  of  some  tasks  are 
performed  simultaneously,  that  is,  components  must  be  time-shared. 

Partitioning  the  components  of  time-shared  tasks  for  part  training  has  been 
termed  fractionation  (Naylor,  1962).  A  number  of  researchers  (e.g.,  Knerr  et 
al.,  1986;  Wickens,  1989)  have  argued  that  fractionation  should  be  designed  in 
accordance  with  the  attentional  resources  of  the  performer.  A  multiple 
resources  conception  of  attention  holds  that  the  elements  of  such  time-shared 
subtasks  must  compete  for  a  limited  pool  of  information  processing  resources. 
Information  processing  resources  are  hypothetical  constructs  in  attention 
referring  to  the  limited  amount  of  "mental  energy"  that  a  performer  has  to 
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apply  to  a  task  (Wickens,  1989).  The  other  key  aspect  to  the  theory  is  that 
there  are  multiple  sources  of  this  energy  and  that  competition  for  this  energy 
occurs  within  but  not  between  resources.  In  other  words,  if  two  subtasks  tap 
different  resources,  they  are  said  to  be  independent  and  can  be  effectively 
performed  together.  To  the  extent  that  subtasks  tap  the  same  resources, 
performers  must  learn  to  time-share  their  attention  among  the  subtasks.  Thus, 
subcomponents  of  a  time-shared  task  are  related  to  the  extent  that  they  tap 
the  same  or  similar  processing  resources. 

Methods.  The  concept  of  multiple  resources  has  evolved  from  empirical 
findings  indicating  the  extent  to  which  tasks  can  be  effectively  time-shared 
and  other  lines  of  converging  evidence.  Wickens  (1989)  suggested  that  global 
task  characteristics  can  be  used  to  predict  the  results  from  these  studies  and 
thus  define  the  different  resource  systems.  These  critical  task 
characteristics  can  be  described  in  terms  of  three  dimensions:  (a)  stage  of 
processing  (encoding,  central  processing,  responding),  (b)  processing  codes 
(analog/spatial  vs.  verbal/linguistic),  and  (c)  processing  modalities 
(auditory  vs.  visual  perception,  and  vocal  vs.  manual  responses).  Wickens 
predicts  that  ".  .  .  two  tasks  that  share  common  levels  on  the  dimensions  will 
suffer  greater  interference  than  two  that  demand  separate  levels"  (p.  82). 

This  qualitative  three-dimensional  model  of  multiple  resources  provides,  in 
essence,  heuristic  guidelines  for  organizing  components  of  a  time-shared  task. 

Man§  and  his  colleagues  (Man§,  Coles,  Karis,  Strayer,  &  Donchin,  1984; 
Man§,  Coles,  Wickens,  &  Donchin,  1983)  developed  empirical  procedures  for 
determining  these  relationships  among  subtasks  of  complex  time-shared  tasks 
based  on  Sternberg's  Additive  Factors  approach.  According  to  their  empirical 
approach,  if  the  effects  of  two  independent  variables  on  performance  are 
additive,  the  factors  must  be  affecting  independent  (i.e.,  different) 
information  processes.  On  the  other  hand,  if  the  effects  are  interactive, 
they  are  related  in  their  effect  on  a  common  information  processing  resource. 
Their  prototypical  analysis,  which  focused  on  a  complex  and  interactive 
computer  game  (Space  Fortress),  identified  three  skill  components:  appraisal, 
motor,  and  perceptual -motor.  Only  appraisal  and  motor  skill  components  were 
shown  to  interact,  but  only  under  speeded  response  conditions. 

Based  on  the  results  from  their  previous  analysis,  Man§,  Adams,  and 
Donchin  (1989)  designed  three  drills  that  were  intended  to  provide  practice  on 
each  of  the  three  components  identified  earlier.  Subjects  in  the  part¬ 
training  group  practiced  these  skill  components  individually  prior  to 
performing  the  whole  task.  After  part  training,  subjects  were  transferred  to 
the  whole-task  (control)  condition  and  run  to  a  performance  criterion. 

Results  indicated  that  the  part-training  group  reached  the  criterion  more 
quickly  than  did  the  control  group  who  practiced  only  under  whole-task 
training  conditions.  In  fact,  the  time  saved  in  the  part-training  group  was 
more  than  the  time  required  for  the  whole-task  training.  In  other  words,  the 
part-training  regimen  resulted  in  greater  than  100%  transfer  to  whole-task 
performance.  To  interpret  these  findings,  Man§  et  al.  suggested  that  the 
superiority  of  part  over  whole  training  was  due  to  the  fact  that  part  training 
allowed  the  subjects  to  study  in  isolation  the  action-outcome  relationships  of 
a  subset  of  task  elements.  These  relationships  were  less  salient  in  the  whole 
training  regimen,  and  performance  consequently  suffered.  In  summary,  the 
results  from  this  research  support  the  notion  that  independent  skill 
components  can  effectively  be  trained  in  isolation. 


9 


The  results  from  Man§  et  al.  (1989)  also  provide  partial  confirmation  of 
the  multiple  resource  conceptions  discussed  above.  To  provide  even  stronger 
evidence  for  the  multiple  resources  concept,  research  should  be  performed  that 
factorial ly  combines  interacting  and  independent  skills  with  part  and  whole 
training.  The  prediction  is  that  whereas  part  training  would  produce  better 
learning  for  independent  skills  (a  replication  of  Man€  et  al.'s  findings), 
whole  training  would  produce  superior  performance  for  interacting  skills.  To 
further  illustrate  this  prediction,  three  basic  but  distinct  tank  commander 
skills  related  to  gunnery  can  be  performed  simultaneously:  (a)  target 
acquisition,  (b)  manipulation  of  commander's  control  handles,  and  (c)  radio 
communication.  The  first  two  skills  (target  acquisition  and  control  handle 
manipulation)  both  require  spatial/analog  coding  of  information  rather  than 
verbal/linguistic  coding  as  required  by  the  third  skill  (radio  communication). 
The  prescription  that  may  be  derived  from  this  qualitative  analysis  of 
processing  requirements  is  that,  whereas  radio  communication  may  be  trained 
apart  from  the  other  two  components,  target  acquisition  and  control  handle 
manipulation  should  be  trained  together.  A  possible  mitigating  variable  is 
level  of  training.  With  high  levels  of  practice,  skills  become  automated  and 
require  a  lower  level  of  processing  resources  (e.g.,  Salmoni,  1989).  It  is 
possible,  then,  that  high  levels  of  part  training  may  be  effective  for 
interacting  skills  if  they  are  sufficiently  automated.  In  summary,  these 
predictions  suggest  a  three-way  interaction  exists  among  training  strategy 
(part  vs.  whole),  relation  between  skills  (independent  vs.  interacting),  and 
level  of  training  (low  vs.  high). 

Evaluation.  The  research  findings  from  controlled  experiments  provide 
limited  support  to  the  view  that,  to  the  extent  time-shared  skills  tap 
qualitatively  different  resources,  they  can  be  successfully  trained  in 
isolation.  This  view  also  implies  that,  to  the  extent  time-shared  skills  tap 
the  same  resource,  they  must  be  trained  together  so  that  the  trainee  learns 
the  appropriate  attentional  skills.  It  should  be  cautioned  that  the  findings 
are  relatively  new,  and  these  methods  have  not  been  applied  to  "real  world" 
jobs.  Thus,  the  suggestions  for  organizing  training  arising  from  attentional 
considerations  should  be  regarded  as  tentative,  and  perhaps  topics  for 
research  in  their  own  right. 

Two  limitations  to  implementing  these  methods  are  apparent.  First,  many 
gunnery  skills  are  strictly  sequential  and,  therefore,  not  relevant  to  the 
analysis  of  multiple  resources.  The  most  relevant  aspect  of  the  gunnery 
domain  would  appear  to  be  the  skills  performed  by  the  tank  commander  who  must 
time-share  his  attention  between  procedural  gunnery  tasks  and  command/control 
tasks.  Second,  the  empirical  analyses  performed  by  Man§  and  colleagues  would 
probably  be  impractical  for  most  gunnery  applications,  other  than  for  research 
purposes.  On  the  other  hand,  as  demonstrated  by  the  preceding  example, 
Wicken's  conceptual  description  of  multiple  resources  may  serve  as  a  useful 
heuristic  guideline  to  the  developer  of  training  strategies. 

Summary  of  Recommendations  for  Structuring  Training 

The  strategies  related  to  structuring  training  were  discussed  in  terms 
of  relationships  existing  among  objectives.  To  the  extent  that  the  relations 
can  be  objectively  measured,  algorithmic  procedures  (e.g.,  cluster  analyses) 
can  be  used  to  organize  the  objectives.  In  fact,  the  research  by  Morrison 
(1984)  indicated  that  these  approaches  are  possible  for  defining  commonality 
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in  task  subgoals.  However,  for  the  most  part,  the  current  state  of  the  art 
does  not  support  objective  criteria  for  the  relations  that  exist  among 
training  objectives.  Therefore,  the  best  methods  remain  heuristic  guidelines. 
The  resulting  best  methodological  guidance  can  be  summarized  in  terms  of  the 
following  heuristic. 

Any  pair  of  objectives  should  be  trained  together  given  that  one  of 
these  four  conditions  exists: 

1.  one  objective  is  dependent  on  the  other,  that  is,  it  has  the  other 
as  a  prerequisite); 

2.  one  objective  supports  the  other,  that  is,  it  transfers  to  the 
other; 

3.  both  objectives  are  sequentially  related  and  share  a  common  task 
subgoal;  or 

4.  both  objectives  are  time-shared  and  share  a  common  attentional 
resource. 

Sequence  Training 

The  structures  identified  by  grouping  cognate  objectives  do  not  yet 
incorporate  recommendations  for  the  order  in  which  the  grouped  training  units 
should  be  presented,  although  such  sequence  effects  have  a  potential  influence 
on  training  effectiveness.  The  optimum  sequencing  of  training  must  take  into 
account  the  relationships  among  objectives,  and  among  the  media/devices  that 
support  those  objectives.  Given  that  training  has  been  structured  into 
separate  instructional  units,  there  are  at  least  two  levels  at  which  the 
problem  of  sequencing  may  affect  training.  First,  the  efficiency  or 
effectiveness  of  training  may  vary  as  a  function  of  the  sequencing  of 
instruction  within  training  units.  For  example,  within  a  unit  of  instruction 
related  to  target  engagement  there  may  exist  sequences  other  than  the  natural 
order  which  would  maximize  the  impact  of  instruction.  Second,  sequencing  may 
also  have  important  effects  between  clusters  of  training  units.  For  instance, 
there  may  be  better  reasons  for  carrying  out  practice  on  precision  gunnery 
before  degraded  mode  gunnery.  In  addition,  it  can  be  argued  that  sequencing 
issues  arise  in  connection  with  the  use  of  the  training  devices  to  which 
segments  of  training  are  allocated,  separately  from  the  nature  of  the  training 
content  itself. 

It  is  unlikely  that  the  literature  will  provide  definitive  answers  to 
many  of  these  sequencing  problems,  as  these  have  not  typically  formed  the 
subject  of  empirical  research.  However,  there  are  a  number  of  general 
principles  that  should  offer  broad  guidance  for  provisional  recommendations. 
Branson  et  al.  (1975)  made  the  points  that  (a)  sequencing  effects  are  long- 
range,  and  unlikely  to  be  revealed  before  end-of-course  assessment; 

(b)  sequencing  is  important  to  low-aptitude  students;  (c)  sequencing  is 
important  with  unfamiliar  materials;  and  (d)  sequencing  is  important  to  non- 
redundant  materials.  The  ISD  document  provides  recommendations  for  dealing 
with  learning  objectives  with  dependent  relationships,  supportive 
relationships,  or  independent  relationships,  using  the  logic  of  the  subject 
matter  as  the  basis  for  classification.  In  general,  the  majority  of  the 
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published  recommendations  concerning  sequencing  (e.g.,  Gagne,  1967;  Merrill, 
1987;  Reigeluth  &  Curtis,  1987;  Ryder,  Beckchi,  Redding,  &  Edwards,  1988)  have 
been  based  on  relationships  within  the  subject  matter.  Since  many  of  these 
relationships  are  hierarchical  in  nature,  these  recommendations  will  be 
reviewed  under  the  general  heading  of  "Hierarchy. 11  However,  there  appear  to 
be  several  considerations  deriving  from  practical  and  experimental  research, 
extraneous  to  the  nature  of  the  subject  matter  to  be  trained,  that  should  also 
play  a  part  in  determining  sequencing. 

Some  instances  of  such  sequencing  principles  are  already  incorporated  in 
Army  training  practice.  Morrison  (1985)  reported  the  following 
recommendations  for  sequencing  within  clusters  from  the  Training  Development 
Handbook  for  the  U.S.  Army  Armor  School: 

1.  Job  performance  order.  Objectives  should  be  trained  in  the  same 
order  that  they  are  performed  on  the  job. 

2.  Chronological  order.  Objectives  corresponding  to  events  should  be 
trained  in  the  same  order  as  they  occur. 

3.  Cause  and  effect.  Instruction  on  the  causes  of  an  event  should 
precede  a  description  of  the  effects  of  an  event. 

4.  Critical  sequence.  Any  critical  sequence  in  a  procedure  must  be 
trained  in  the  order  the  sequence  occurs. 

5.  Simple  to  complex.  Simple  objectives  should  be  trained  prior  to 
complex  ones. 

6.  Known  to  unknown.  Familiar  topics  should  be  handled  before 
unfamiliar  ones. 

It  is  suggested  that  the  same  recommendations  should  be  applied  to 
sequencing  between  clusters,  with  the  additional  suggestion  that  it  may  be 
possible  to  follow  the  order  exhibited  by  a  tactical  scenario.  The  above 
recommendations  suggest  several  other  main  categories  for  the  analysis  of 
sequencing.  For  the  present  purposes,  it  appears  sufficient  to  group  together 
the  principles  of  job  performance  order,  chronological  order,  and  critical 
sequence  as  instances  of  "Natural  Order."  Following  a  tactical  scenario  may 
be  regarded  as  a  further  instance  of  the  same  type.  Army  practice  also 
sanctions  a  principle  known  as  "Crawl-Walk-Run, "  which  deserves  separate 
consideration  on  its  own  merits.  The  list  above  contains  three  further 
principles,  including  cause  and  effect,  simple  to  complex,  and  known  to 
unknown.  Cause  and  effect  will  not  be  considered  below,  since  there  appears 
to  be  no  strong  backing  for  this  rule.  In  fact,  in  teaching  medicine  it  is 
not  uncommon  for  symptoms  to  be  learned  before  the  etiology  of  diseases.  The 
known  to  unknown  principle  reflects  the  usual  educational  practice,  and  may  be 
subsumed  with  the  other  instructional  recommendations  under  the  general 
heading  of  "Hierarchy."  However,  the  simple  to  complex  recommendation  raises 
issues  connected  with  asymmetrical  transfer  of  training  between  easy  and 
difficult  tasks,  and  will  be  reviewed  under  the  section  entitled  "Task 
Difficulty."  In  summary,  three  important  categories  for  seauencing 
instruction  are  (a)  Natural  Order,  (b)  Crawl-Walk-Run,  and  (c)  Task 
Difficulty. 
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In  addition  to  the  categories  already  mentioned,  two  further  types  of 
principles  deserve  consideration.  The  first  of  these  is  the  principle  of 
"Chaining",  which  is  derived  from  the  body  of  research  on  part- task  training. 
The  other  is  a  principle  entitled  "Alternation,"  which  in  this  connection 
refers  primarily  to  the  practice  of  giving  varied  exposure  to  different 
devices.  The  evidence  for  such  a  principle  is  mainly  indirect,  but  logically 
compelling.  The  principles,  or  groups  of  principles,  to  be  evaluated  are 
summarized  in  Table  1.  The  table  shows  the  general  titles  used,  the  area  of 
research  or  practice  from  which  these  principles  derives  support,  and  the 
level  of  application  of  the  principles  (between  training  units;  between 
devices;  within  training  units).  All  five  groups  of  principles  are  discussed 
in  more  detail  in  the  following  sections. 


Table  1 


Principles  of  Sequencing 


Principle 

Original  Context 

Level  of  Application 

Hierarchy 

Cognitive  structure 

B,  D,  W 

Natural  Order 

Standard  procedure 

B,  W 

Crawl-Walk-Run 

Army  practice 

B 

Task  Difficulty 

Transfer  of  training 

B,  W 

Chaining 

Part-task  training 

B,  W 

Alternation 

Simulator  fidelity 

D 

Note.  B  =  Between  units;  D  =  Between  devices;  and  W  =  Within  units. 


Hierarchy 

Various  approaches  to  motor  control  and  learning  incorporate 
hierarchical  principles  of  cognitive  organization.  Thus,  most  current 
theories  of  skill  make  the  assumption  that  control  is  exerted  through  at  least 
two  levels.  For  example,  an  idea  common  to  theories  of  typing,  writing, 
speech  production,  and  musical  performance  is  that  a  higher  level  of  control 
determines  the  nature  of  a  given  activity,  while  a  lower  level  governs  the 
execution  of  motor  commands  (Colley,  1989).  The  implications  for  training, 
were  examined  by  Mackay  (1982),  who  concluded  that  transfer,  although 
relatively  specific,  occurs  at  the  higher  levels  of  organization.  However, 
such  theories  appear  to  have  no  direct  implications  for  the  sequencing 
problem.  What  is  clear  from  this  hierarchical  perspective  is  that  the 
programming  of  response  sequences  must  normally  precede  their  execution,  so 
that  control  is  passed  sequentially  from  higher  to  lower  levels.  On  the  other 
hand,  it  cannot  be  assumed  that  training  objectives  should  follow  the  same 
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order.  In  fact,  hierarchical  theories  usually  hold  that  lower  levels  should 
be  learned  prior  to  higher  ones. 

Methods.  It  may  appear,  at  first,  that  applying  the  notion  of  hierarchy 
to  the  principles  of  instruction  should  make  an  important  contribution.  Gagn§ 
(1967),  for  example,  has  long  argued  that  learning  takes  place  at  a  number  of 
different  levels  of  generality.  Specifically,  he  proposed  that  a  sequence  is 
imposed  by  the  requirement  that  acquisition  at  each  level  is  dependent  on 
prior  mastery  of  the  underlying  (prerequisite)  level.  Thus,  a  high-level 
skill  such  as  problem-solving  assumes  a  mastery  of  rule  using,  classifying, 
multiple  discrimination,  verbal  and  motor  chaining,  and  specific  responding 
(in  that  order)  with  mastery  at  each  level  imposing  different  preconditions. 
Without  accepting  this  analysis  in  full,  there  is  no  doubt  that  many  subject 
matters  dictate  their  own  logical  progression.  However,  it  is  by  no  means 
clear  that  tank  gunnery  requires  all  of  these  levels.  In  any  case,  the  most 
natural  application  would  be  within  training  units,  so  that  the  analysis 
provides  little  guidance  for  sequencing  between  units. 

Many  authors  concerned  with  structuring  subject  matter  have  employed 
versions  of  schema  theory  in  conjunction  with  hierarchical  concepts.  Morris 
and  Rouse  (1985)  considered  and  evaluated  several  training  methods  for 
troubleshooting,  coming  to  the  conclusion  that  the  most  efficient  technique  is 
one  in  which  trainees  are  specifically  taught  to  make  lower-level  applications 
of  general  theoretical  principles.  As  Ryder  et  al.  (1988)  pointed  out,  this 
method  of  teaching  prototypical  concepts  first,  followed  by  variations, 
structures  new  information  in  a  way  that  can  most  readily  be  related  to 
existing  schemata.  The  general  sequencing  model  described  by  Reigeluth  and 
Curtis  (1987)  depends  on  providing  opportunities  for  integrating  applications 
and  theory  by  using  forms  of  progressive  elaboration.  This  approach  is  where 
training  begins  with  the  shortest  path  through  the  network  of  possible 
procedures;  procedural  branches  (elaborations)  are  systematically  added  until 
the  entire  network  is  learned.  However,  the  method  of  choice  may  depend  on 
the  volume  of  material  to  be  covered.  Merrill  (1987)  favored  what  he  terms 
the  "hierarchical  part-whole"  relationship  among  component  parts  of  a  task  to 
be  taught,  but  considers  the  exception  when  a  task  can  be  taught  in  one 
instructional  session.  In  these  circumstances,  a  major  step  in  the  hierarchy 
and  its  substeps  may  be  taught  at  the  same  time. 

In  any  case,  the  notion  of  hierarchy  as  applied  directly  to  memory 
organization  seems  potentially  capable  of  producing  more  specific 
recommendations.  Morrison  (1984)  demonstrated  the  application  of  proximity 
analysis  to  clustering  in  the  memory  scores  of  armor  crewmen  for  the 
procedural  elements  of  a  machine  gun  task,  which  required  performance  in  a 
fixed  order,  and  in  a  tank  radio  task,  which  did  not.  In  both  cases,  the 
combination  of  clustering  techniques  with  rational  analyses  of  goals  and 
subgoals  gave  rise  to  interpretable  tree  structures.  In  this  case,  however, 
the  derived  structure  for  the  gunnery  task  shows  levels  of  control  such  as 

(a)  "clearing  the  M240"  requires  "unload"  and  "return"  operations, 

(b)  "unload"  in  turn  requires  "charge  bolt,"  "open  receiver,"  and  "remove 
ammo,"  and  finally  (c)  "open  receiver"  requires  "place  in  SAFE"  and  "open 
cover."  These  phases  of  operation  do  not  necessarily  correspond  to  successive 
training  requisites.  In  fact,  Hoffman  et  al.  (1983)  compared  top-down  and 
bottom-up  training  strategies  using  similar  memory  organization  analyses  and 
failed  to  find  differences  between  the  two  approaches  to  sequencing. 
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Hoffman  and  Morrison  (1988)  made  rational,  hierarchical  analyses 
directly  targeted  at  identifying  the  prerequisite  skills  and  the  instructional 
sequence  for  tank  gunnery  training.  The  analysis  suggested,  as  indicated 
above,  that  gunnery  consists  of  a  large  number  of  relatively  simple  subtasks. 
Hence,  the  structure  of  the  gunnery  domain  may  be  described  as  "wide,  but  not 
very  deep."  If  it  is  the  case  that  gunnery  consists  of  many  disparate 
activities,  each  of  which  relies  rather  little  on  prerequisite  skills  and 
knowledge,  the  contribution  of  a  hierarchical  analysis  to  sequencing  problems 
will  be  relatively  slight.  Nevertheless,  the  associated  principle  of 
progressive  elaboration,  which  suggests  that  the  learning  of  a  core  activity 
should  precede  the  learning  of  more  complex  versions  of  the  task,  should  be 
borne  in  mind. 

Evaluation.  Although  hierarchical  principles  do  not  provide  a  complete 
or  automatic  solution  to  sequencing  problems,  it  is  possible  to  identify  some 
specific  applications  of  the  notion  that  certain  instances  of  learning  should 
precede  others.  Between  training  units,  for  example,  there  may  be  a  logical 
case  for  learning  tank  commander  duties  before  attempting  to  learn  the 
coordination  between  tank  commander  and  gunner.  Again,  although  there  is  some 
doubt  as  to  how  the  hierarchy  principle  should  be  applied,  it  may  be  possible 
to  argue  that  some  degraded  mode  techniques  represent  more  primary  skills,  and 
that  these  should  be  taught  before  practice  on  some  non-degraded  mode 
techniques.  However,  although  some  degraded  mode  techniques  may  be  more 
primary  from  a  technical  standpoint,  the  skills  required  are  probably  more 
demanding.  With  regard  to  sequencing  between  devices,  it  must  be  determined 
which  of  the  devices  trains  the  more  basic  skills. 

Natural  Order 


The  recommendation  that  training  should  follow  the  natural  order  of  the 
task  or  subject  matter  to  be  trained  is  an  obvious  and  implicit  rule  for 
instructional  sequencing.  Thus,  it  may  be  considered  the  "default"  sequence 
for  training. 

Methods.  As  already  indicated  (Morrison,  1985),  the  natural  order 
principle  may  be  applied  at  several  levels.  With  respect  to  specific  jobs, 
training  objectives  should  be  sequenced  in  the  order  that  they  are  normally 
performed.  In  a  number  of  cases,  no  practical  alternative  exists  because  the 
job  order  constrains  the  logical  sequence  of  training.  Thus,  when  preparing  a 
defensive  position,  an  instructional  objective  entitled  "Select  a  Firing 
Position"  should  be  trained  before  the  objective  entitled  "Prepare  a  Sketch 
Range  Card,"  because  the  tank  must  first  be  in  a  firing  position  before  the 
crew  can  prepare  a  sketch  range  card.  The  alternative  approach  of  preparing 
or  simulating  an  artificial  firing  position  in  order  to  allow  range  card 
practice  can  only  be  justified  in  special  circumstances. 

The  more  general  guideline  concerns  the  chronological  order  of  events, 
suggesting  that  objectives  should  be  trained  in  the  same  order  as  the 
corresponding  events  normally  occur.  Thus,  the  activity  "Maintain  Position  in 
Platoon  Formation"  takes  place  after  the  initiation  of  "Conduct  Tactical  Road 
March,"  and  the  corresponding  training  objectives  should  be  sequenced  in  the 
same  order.  The  initiation  of  the  road  march  activity  presents  the  cues 
needed  to  initiate  the  position  maintenance  activity,  so  that  the  first 
activity  naturally  leads  into  the  other.  Again,  certain  activities  are 
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physical  precursors  of  other  activities,  or  else  constitute  strongly  coercive 
preconditions.  If  the  required  activity  is  to  charge  a  machine  gun,  the 
safety  must  be  off  before  pulling  the  bolt  to  the  rear.  Hence,  it  is  clear 
that  training  should  be  administered  with  reference  to  putting  the  safety  in 
the  "Fire"  position  before  training  the  charging  procedure.  In  this  form,  the 
guiding  principle  is  that  of  critical  sequence. 

Evaluation.  The  recommendations  of  this  section  are  relatively  clear, 
although  it  is  possible  to  contemplate  some  exceptions.  For  example,  the 
principle  of  "backward  chaining"  (see  below)  would  override  the  requirements 
of  natural  order  if  shown  applicable  to  the  objectives  otherwise  governed  by 
the  principle  of  natural  order.  However,  given  the  relative  lack  of  evidence 
favoring  backward  chaining  as  against  other  directions  of  part-task 
sequencing,  there  is  currently  no  reason  to  avoid  following  job  performance 
order,  chronological  order,  and  critical  sequence  order. 

Crawl-Walk-Run 


A  traditional  Army  approach  recommends  that  trainees  should  first  learn 
to  crawl,  then  to  walk,  and  finally  to  run.  Although  this  principle  seems  to 
encourage  gradually  increasing  demands  on  the  trainee,  its  exact 
interpretation  may  take  several  forms,  some  of  which  are  described  below. 

Methods.  The  crawl-walk-run  method  appears  at  first  sight  to  reflect  a 
concern  for  training  accuracy  while  speed  is  gradually  incremented.  Accuracy 
first  and  speed  later  typifies  the  acquisition  of  many  skills,  although  it  is 
sometimes  objected  that  speed  changes  the  form  of  response  so  that  accuracy 
later  should  be  preferred.  The  older  work  on  speed  versus  accuracy  training 
indicates  that  the  issue  is  relatively  unimportant  (Holding,  1965),  because 
there  are  few  tasks  whose  manner  of  performance  is  radically  altered  when 
practiced  at  different  speeds.  In  fact  Poffenberger 1 s  Law,  sometimes  invoked 
in  industrial  psychology,  states  that  any  task  initially  learned  to  an 
accuracy  criterion  can  later  be  performed  at  successively  greater  speeds. 

There  are  a  number  of  exceptions  to  this  generalization  due  to  special 
considerations  such  as  response  timing,  pacing,  or  sharing.  However,  none  of 
these  exceptions  has  any  established  relevance  to  sequencing. 

A  different  interpretation  of  the  principle  is  reported  by  Drucker  and 
Morrison  (1987),  who  described  the  crawl  stage  as  an  orientation  phase, 
perhaps  with  verbal  instruction  and  visual  demonstration;  the  walk  phase  as 
consisting  of  task  practice,  but  in  isolation  from  other  activities;  and  the 
run  phase  as  integrated  performance  in  realistic  contexts.  This 
interpretation  is  somewhat  reminiscent  of  Fitts'  (1964)  sequence  for  skill 
acquisition,  which  comprises  cognitive,  associative,  and  automatic  phases. 
However,  the  crawl-walk-run  formula  also  seems  to  imply  a  progression  from 
easy  to  difficult  conditions  of  practice  (see  below),  together  with  the  use  of 
increasingly  realistic  contexts.  The  principle,  as  expanded  by  Drucker  and 
Morrison,  further  implies  that  training  objectives  should  be  learned 
repetitively.  This  should  result  in  a  degree  of  overlearning,  which  in  turn 
should  provide  some  resistance  to  stress  effects.  In  addition,  the  repetition 
should  make  it  possible  to  expose  the  learner  to  a  variety  of  different 
practice  materials,  a  factor  which  Wolfle  (1951)  already  recognized  as 
contributing  to  successful  training.  Hence,  the  principle  may  also  be  related 
to  the  recommendation  for  alternation  discussed  below. 
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Evaluation.  None  of  the  above  interpretations  seems  in  any  way 
objectionable.  However,  the  principle  is  apparently  too  complex  to  offer 
clear  guidance  with  regard  to  sequencing,  although  it  does  provide  several 
useful  pointers.  Among  these,  note  that  the  gradual  increase  in  speed,  the 
implied  repetition,  and  the  increase  in  realism  are  all  most  naturally  applied 
between  rather  than  within  training  units.  The  principle  overlaps  with  others 
discussed  below,  including  the  training  recommendations  based  on  task 
difficulty  and  alternation  of  practice. 

Task  Difficulty 

As  indicated  above,  the  crawl-walk-run  approach  may  be  interpreted  as 
implying  an  easy-to-diff icult  direction  for  training,  if  not  an  explicit 
progression  from  slow  to  fast.  In  any  case,  the  easy-to-diff icult  sequence 
deserves  consideration  in  its  own  right. 

Methods.  The  normal  educational  sequence  is  to  proceed  gradually  from 
easier  to  more  difficult  aspects  or  versions  of  the  task  to  be  learned,  just 
as  from  the  known  to  the  unknown.  This  form  of  progression  is  basically 
compatible  with  the  principles  of  programmed  learning,  which  in  turn  derive 
support  from  the  behavior  shaping  literature.  As  Holland  (1960)  reiterated, 
the  programmed  learner  should  be  led  by  gradual  stages  from  what  he  knows  at 
the  outset  toward  the  goal  of  the  training.  Each  new  step  brings  a  small 
increase  in  complexity,  so  that  the  learner  can  readily  master  the  new 
material  and  therefore  achieve  a  high  rate  of  success. 

A  potential  complication  is  that  the  progression  of  learning  from  one 
trial,  exercise,  or  session  to  another  may  be  viewed  as  a  form  of  successive 
transfer  of  training.  Because  there  are  some  examples  in  the  literature  where 
better  transfer  of  training  occurs  in  the  diff icult-to-easy  direction,  it  is 
sometimes  assumed  that  this  may  be  the  superior  order  for  practice  in  a 
training  program.  In  fact,  there  are  anecdotal  accounts  of  medical  students 
learning  to  perform  surgical  operations  using  the  left  hand  (the  more 
difficult  version),  on  the  assumption  that  there  would  later  be  automatic 
transfer  to  the  (easier)  right.  However,  it  is  easier  to  find  examples  in  the 
easy-to-diff icult  direction. 

In  general,  it  appears  that  perceptual  tasks  give  rise  to  better  easy- 
to-diff  icult  transfer,  as  do  changes  in  target  speed  and  amplitude  in  tracking 
studies  (Holding,  1987).  The  opposite  result  (i.e.,  better  diff icult-to-easy 
transfer)  tends  to  occur  with  other  forms  of  task  complexity,  with  changes  in 
control  characteristics,  and  with  reversals  of  display-control  relationships. 
However,  small  changes  in  task  variables  can  produce  transfer  in  both 
directions  within  the  same  experimental  paradigm.  It  appears  that  the  results 
can  be  explained  in  terms  of  the  tradeoff  between  two  opposing  tendencies: 

One  tendency  is  the  principle  of  inclusion,  which  results  in  the  trainee 
learning  more  (i.e.,  more  habits,  skills)  on  the  difficult  version  of  a  task; 
the  other  is  the  principle  of  performance  standards,  which  leads  to  the 
trainee  learning  better  response  habits  on  the  easier  version. 

Inclusion  takes  place,  for  example,  when  a  trainee  learns  to  track  a 
difficult  target  course  whose  bandwidth  literally  includes  the  frequencies 
used  in  the  easier  target  course.  The  operation  of  the  inclusion  principle 
was  been  verified  in  a  complex  tracking  task  by  Williges  and  Baron  (1973), 
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although  there  appear  to  be  few  instances  in  which  such  a  principle  is 
applicable  to  armor  training.  On  the  other  hand,  the  operation  of  the 
performance  standards  principle  appears  to  explain  the  recent  results  of 
Lintern,  Roscoe,  and  Sivier  (1989).  These  researchers  found  better  transfer 
to  a  flight  task  with  crosswind  by  groups  learning  without  crosswind  (easy) 
than  by  those  learning  with  a  crosswind  (difficult),  despite  the  fact  that  the 
criterion  task  was  identical  to  the  difficult  version.  In  any  case,  there  may 
be  a  substantial  difference  in  the  time  required  for  original  learning  in  the 
two  directions  of  transfer  regardless  of  the  optimal  direction  for  transfer 
itself. 

It  is  important  to  note  that  the  optimum  direction  of  transfer  may  not 
be  the  best  direction  for  training.  That  is,  one  cannot  equate  transfer 
percentage  with  training  time.  Even  with  better  transfer  in  the  diff icult-to- 
easy  direction,  the  combined  training  times  for  the  two  versions  may  exceed 
the  training  time  in  the  reverse  direction.  Table  2  gives  a  hypothetical 
illustration  of  this  effect.  Learning  to  play  the  organ  to  some  criterion  may 
take  200  hours,  dropping  to  140  after  first  learning  the  piano.  There  is  thus 
a  saving  of  60  hours,  out  of  200,  giving  a  transfer  value  of  30  percent.  In 
the  other  direction,  the  piano  learning  time  drops  from  100  to  60  after  the 
organ,  for  a  transfer  value  of  40  percent.  Organ-to-piano,  the  diff icult-to- 
easy  combination,  therefore  shows  superior  transfer.  Nevertheless,  the  most 
efficient  training  sequence  consists  of  progressing  from  piano  to  organ,  since 
the  overall  training  time  for  both  tasks  totals  240  hours,  instead  of  260 
hours.  The  issue  of  total  time  is  an  important  factor  in  allocating  training 
time,  which  is  discussed  in  more  detail  in  the  section  concerning  that 
particular  problem. 


Table  2 

Asymmetric  Transfer  versus  Training  Sequence 


Sequence 


(Piano  -)  Organ  (Organ  -)  Piano 


Hours 

(Percent) 

Hours 

(Percent) 

As  Control  (1st)  Task 

200 

100 

As  Transfer  (2nd)  Task 

140 

60 

Savings 

Transfer  Value 

60 

(30%) 

40 

(40%) 

Both  Tasks 

240 

260 

Evaluation.  Obtaining  better  transfer  from  first  learning  a  difficult 
task  is  clearly  of  no  benefit  if  the  difficult  task  itself  requires  an 
extended  learning  period.  The  majority  of  applied  training  studies,  like 
those  in  education,  seem  to  concur  in  finding  the  easy-to-diff icult  direction 
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preferable.  The  recommendation  is  to  move  from  easy  to  difficult  provided 
that  the  potential  for  direct  inclusion  is  absent. 

Chaining 

Backward  and  forward  chaining  are  techniques  of  part-task  training  for 
skilled  tasks  that  (a)  are  amenable  to  segmentation,  and  (b)  exhibit 
sequential  properties.  When  appropriate,  the  chaining  method  has  implications 
for  both  the  order  and  the  amount  of  practice  on  task  segments.  In  a  task 
divided  into  3  segments  (A,  B,  C),  backward  chaining  would  refer  to  practicing 
first  C,  then  BC,  then  ABC;  forward  chaining  would  refer  to  practicing  first 
A,  then  AB,  then  ABC.  Note  that  backward  chaining  thus  entails  more  practice 
on  segment  C,  and  forward  chaining  requires  more  practice  on  segment  A.  The 
advantage  of  chaining  is  that  practicing  overlapping  segments  should  help  to 
integrate  the  sequence,  thereby  overcoming  the  problem  that  extra  trials  may 
be  needed  for  reintegration  after  part  practice. 

Method.  The  chaining  concept  has  no  application  in  a  task,  such  as 
dart-throwing,  for  which  segmentation  is  impracticable.  In  such  cases,  it  may 
be  that  other  part-task  training  methods  (e.g.,  task  fragmentation  or 
simplification)  are  more  viable  alternatives  for  part-task  decomposition 
(Wightman  &  Lintern,  1985).  However,  these  methods  have  no  relevance  to  the 
sequencing  problem.  It  also  appears  that  chaining  methods  are  irrelevant  to 
tasks  in  which  the  order  of  practice  of  the  segments  is  unimportant,  such  as 
the  identification  of  a  set  of  targets.  A  case  in  point  was  reported  by 
Singer  (1968),  who  examined  the  acquisition  cf  skill  at  volleyball.  Part-task 
practice  on  the  component  subskills  (serving,  digging,  setting  and  spiking) 
was  superior  to  whole-task  training,  but  the  order  in  which  these  subskills 
were  learned  had  no  effect  on  the  final  outcome.  Man§  et  al.  (1989)  have  also 
obtained  superior  performance  for  part-task  training  compared  to  whole-task 
training  without  specific  chaining.  On  the  other  hand,  experiments  on 
aircraft  simulator  training  on  tasks  that  are  more  clearly  sequential  in 
nature  (e.g.,  Bailey  et  al.,  1980)  have  obtained  apparent  performance 
advantages  for  part-task  training  with  backward  chaining. 

The  task  components  in  volleyball  do  appear  to  preserve  a  modicum  of 
natural  order,  but  it  is  evident  that  a  stronger  ordering  must  be  required  if 
any  chaining  effect  is  to  be  observed.  Hence,  it  may  be  argued  that  the 
chaining  principle  is  more  likely  to  find  application  within  the  instructional 
units  for  tank  gunnery,  where  the  ordering  is  more  coercive,  than  between 
these  units.  Within  a  unit  on  target  engagement,  for  example,  there  appears 
to  be  a  strong  ordering  of  components  such  as  rotating  the  turret,  aligning 
the  sights,  squeezing  the  trigger,  and  then  observing  the  outcome.  Between 
units,  on  the  other  hand,  an  ordering  may  be  imposed  by  the  need  for  target 
identification  to  precede  marksmanship,  but  it  seems  dubious  whether  this 
degree  of  ordering  satisfies  the  necessary  conditions  for  chaining. 

For  the  cases  where  chaining  is  appropriate,  it  should  be  noted  that  the 
apparent  advantages  of  the  backward  form  of  chaining  may  be  unreliable.  In 
the  earlier  work  on  flight  simulators  reviewed  by  Wightman  and  Lintern  (1985), 
the  effects  of  segment  order  were  confounded  with  the  effects  of  segment 
difficulty.  Specifically,  the  problem  arises  because  the  final  segments  of 
such  tasks  as  aircraft  landing  were  practiced  first,  and  practiced  most  often, 
when  backward  chaining  was  in  place.  These  segments  were  also  the  most 
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difficult,  and  were  the  segments  that  resulted  in  the  greatest  amount  of 
knowledge  of  results.  These  findings  should  be  contrasted  with  the  data 
reported  by  Ash  and  Holding  (in  press)  where  difficulty  and  segment  order  were 
systematically  counterbalanced.  The  latter  work,  contained  in  a  dissertation 
by  Ash  (1987),  used  a  keyboard  task  of  musical  performance.  The  use  of  music 
as  the  medium  is  less  relevant  than  the  fact  that  the  task  involved  a  form  of 
keyboard  encoding  and  translation.  In  fact,  the  musical  stimuli  conferred  a 
degree  of  natural  ordering,  and  made  it  possible  to  counterbalance  the 
ordering  of  difficult  and  easy  segments  of  the  task.  Knowledge  of  results  was 
held  to  a  minimum  throughout  training.  Backward  and  forward  chaining  were 
compared  with  whole  training  on  (a)  whole-task  criterion  trials  immediately 
following  the  training  procedure,  and  (b)  whole-task  retention  trials  one  week 
later.  Both  forms  of  part-task  training  gave  results  superior  to  the  whole- 
task  training,  but  forward  chaining  led  to  better  learning  than  did  backward 
chaining. 

Evaluation.  It  is  possible  to  argue  that  the  above  results  arose  as  a 
consequence  of  the  special  characteristics  of  the  musical  performance  task, 
which  is  a  discrete-response  task  as  well  as  sequential.  Further  work  is  now 
being  conducted  to  replicate  Ash  and  Holding  (in  press)  using  a  continuous 
task  instead.  However,  it  appears  more  probable  that  the  earlier  results 
favoring  the  backward  direction  were  due  simply  to  the  omission  of  a  forward 
chaining  condition.  Because  both  backward  and  forward  chaining  were  superior 
to  whole-task  training,  there  is  implicit  confirmation  of  the  idea  that 
overlapping  practice  is  beneficial.  There  have  been  no  direct  comparisons  of 

chaining  against  simple  partitioning  methods,  and  it  is  again  clear  that 

further  experimentation  is  required.  In  the  absence  of  contrary  evidence,  it 

can  be  presumed  that  the  overlapping  practice  contained  in  chaining  procedures 
has  beneficial  effects.  With  respect  to  the  direction  of  chaining,  the 
preliminary  conclusion  must  be  that  forward  chaining,  which  has  the  advantage 
of  preserving  the  natural  order  of  the  task,  is  the  superior  alternative. 

This  conclusion  may  require  modification  in  practical  cases  where  segment 
difficulty  and  knowledge  of  results  cannot  be  made  independent  of  task  order. 

Alternation 


As  shown  in  Table  1,  sequencing  by  alternation  is  most  naturally  applied 
to  training  devices,  rather  than  to  objectives.  The  principle  of  alternation 
between  devices  has  no  direct  empirical  backing,  but  represents  a  logical 
response  to  the  problems  of  incomplete  device  fidelity.  Alternating  between 
different  devices  also  provides  the  benefit  of  varied  practice,  and  should 
assist  trainees  in  developing  a  mental  model  of  the  overall  armor  system. 
Possible  interactions  between  the  requirements  of  alternation  and  such 
techniques  as  part-whole  training  have  not  been  investigated. 

Method.  It  has  often  been  confirmed  (most  recently  by  Lintern, 

Sheppard,  Parker,  Yates,  &  Nolan,  1989)  that  strict  physical  fidelity  is  of 
less  importance  than  psychological,  or  functional,  fidelity.  Nevertheless, 
even  small  departures  from  physical  fidelity  will  make  for  some  difficulties 
in  transferring  from  training  equipment  to  the  real  task,  particularly  as  the 
duration  of  training  increases  and  as  learning  consequently  becomes  more 
specific.  Except  when  fidelity  is  perfect,  device-specific  responses  will  be 
found  inappropriate  to  the  real  task.  It  seems  apparent  that  alternating 
between  different  devices,  whose  departures  from  fidelity  may  take 
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complementary  forms,  should  permit  the  trainee  to  build  a  more  adequate 
cognitive  model  of  the  target  skill,  thus  producing  an  effective  increase  in 
psychological  fidelity.  Furthermore,  rotating  between  devices  increases  the 
variety  of  practice  during  training.  The  conclusion  that  device  alternation 
should  be  recommended  therefore  receives  additional  support  from  the 
consideration  that  using  a  variety  of  practice  materials  is  known  to  improve 
learning,  as  mentioned  earlier.  The  variety  principle  is  further  sanctioned 
by  Schmidt's  (e.g. ,  1975)  schema  theory,  and  gives  rise  to  more  general izable 
transfer  effects. 

A  further  step  in  alternation  should  be  to  include  the  real-world 
equipment,  in  this  case  the  Ml  tank,  in  the  rotation  between  devices.  This 
procedure  should  not  only  increase  the  variety  of  training  practice,  but 
should  have  two  further  desirable  consequences.  First,  the  real  equipment 
will  provide  a  stable  reference  for  the  correction  of  misapprehensions  based 
on  minor  deficiencies  of  the  simulator  devices,  which  can  then  be  discounted. 
Second,  early  experience  with  the  real  equipment  helps  to  define  the  goal  of 
the  training  exercise,  ensuring  that  training  takes  place  in  a  functional 
context.  It  is  a  tenet  of  Bandura's  (1986)  theory  of  observational  learning, 
for  example,  that  little  progress  can  be  made  until  the  trainee  constructs  a 
cognitive  model  of  the  target  activity.  Again,  Gray  and  Orasanu  (1987) 
emphasize  the  part  played  by  mental  models  in  training  for  cognitive  skills. 

It  seems  essential  that  trainees  should  fully  "understand"  the  system  in  which 
they  are  to  operate,  building  a  cognitive  model  which  can  be  supplemented  or 
modified  as  further  learning  takes  place. 

Evaluation.  The  above  conclusions  appear  relevant  to  gunnery  training 
objectives,  suggesting  that  exposure  to  varied  devices,  including  the  actual 
tank,  should  change  the  trainees'  semantic  interpretation  of  training 
exercises.  Varied  practice  should  lay  the  foundation  for  more  flexible 
performance  in  later  applications,  with  a  consequent  increase  in  training 
effectiveness.  Including  early  exposure  to  the  real  task  should  help  to 
overcome  deficiencies  in  simulator  fidelity,  and  should  assist  trainees  in 
building  an  adequate  mental  model  of  the  target  system.  However,  this  aspect 
of  sequencing  has  been  insufficiently  researched,  and  there  is  a  clear  need 
for  applied  experimentation  on  this  issue. 

Summary  of  Recommendations  for  Sequencing  Training 

Reviewing  the  available  principles  for  the  sequencing  of  training  units 
has  produced  a  set  of  conclusions  varying  widely  in  degree  of  empirical 
support  and  in  applicability  to  the  problems  of  armor  training.  The  preceding 
analyses  have  screened  out  several  potential  sequencing  principles,  but  there 
remains  a  core  of  rules  which  appear  to  be  provisionally  supported.  It  should 
be  noted  that  no  direct  evidence  supports  the  notion  that  the  sequencing  of 
instructional  units  is  of  greater  importance  than  are  many  other  training 
variables.  Nevertheless,  the  use  of  these  principles  should  play  a  part  in 
optimizing  the  conditions  that  determine  training  effectiveness.  Table  3 
provides  a  brief  summary  of  the  best  supported  recommendations. 

The  analysis  has  thus  given  rise  to  6  groups  of  sequencing  principles, 
distinguishable  into  15  separate  rules.  As  Vineberg  and  Joyner  (1980)  note, 
there  are  many  instances  in  which  such  principles  will  result  in  conflicting 
recommendations,  and  reconciling  the  recommendations  stemming  from  such  an 
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array  of  variables  must  inevitably  involve  a  degree  of  compromise.  However, 
it  will  become  apparent  that  there  are  few  serious  conflicts  between  the 
stated  principles  in  the  context  of  armor  training.  The  applications  section 
of  the  present  report  considers  illustrative  uses  of  the  principles,  showing 
considerable  similarities  between  the  resulting  recommendations  for  the  order 
in  which  practice  should  be  conducted. 


Table  3 

Summary  of  Sequencing  Rules 


1 .  Hierarchy 

1.1  Teach  ends  before  means. 

1.2  Follow  the  logic  of  the  subject  matter. 

1.3  Use  progressive  elaboration. 

2.  Natural  order 

2.1  Follow  job  performance. 

2.2  Follow  chronological  order. 

2.3  Observe  critical  sequence. 

3.  Crawl-walk-run 

3.1  Move  from  slow  to  fast  practice. 

3.2  Move  from  orientation,  through  practice,  to  real  task. 

3.3  Repeat  practice  in  different  modes  (cf.  #6). 

4.  Task  difficulty 

4.1  Transfer  order  is  not  training  order. 

4.2  In  general,  progress  from  easy  to  difficult. 

5.  Chaining 

5.1  Use  overlapping  forms  of  part  practice. 

5.2  Follow  the  natural  sequence  of  the  task  (cf.  #2). 

6.  Alternation 

6.1  Vary  practice  between  training  devices. 

6.2  Provide  real  equipment  experience  early  in  training. 


Select  Media/Devices 

The  third  problem  addressed  by  training  strategies  concerns  the 
selection  of  media  or  devices  for  training.  For  previous  strategies,  we 
decomposed  the  overall  problem  into  multiple  subproblems  and  discussed 
approaches  to  each  of  the  subproblems.  In  this  regard,  Kribs,  Simpson,  and 
Mark  (1983)  provided  a  reasonable  decomposition  of  the  media  selection  problem 
into  three  subproblems:  (a)  determine  the  requirements  of  the  objectives  to  be 
trained,  (b)  determine  the  capabilities  of  training  media,  and  (c)  specify  the 
best  match  of  requirements  and  capabilities.  The  unique  aspect  of  this 
strategy  is  that  media  selection  models  have  been  designed  to  address  all 
three  aspects  of  this  problem  rather  than  any  one  aspect.  Hence,  the 
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following  models  necessarily  overlap  in  function.  The  first  two  sections 
address  some  previous  models  from  the  literature,  and  the  third  proposes  a 
model  based  on  utility  theory  that  has  some  distinct  advantages  over  the 
previous  models. 

Models  Based  on  Analysis  of  Learning  Activities 

One  group  of  older  media  selection  models  can  all  be  traced  to  the 
instructional  theories  of  Robert  Gagn§  and  his  colleagues.  Gagn§  has 
consistently  maintained  (e.g.,  Gagn§,  1984)  that  learning  activities  can  be 
linked  to  certain  types  of  learning  outcomes.  For  instance,  specific  learning 
prescriptions  (e.g.,  telling  students  what  they  are  expected  to  know)  may  work 
for  some  outcomes  (e.g.,  procedural  knowledge),  but  not  for  others  (e.g., 
attitudes).  This  assumption  is  central  to  his  media  selection  procedures  as 
well  as  other  procedures  within  his  model  for  training  development. 

Method.  The  prototypical  example  from  this  group  is  the  model  developed 
by  Braby,  Henry,  Parrish,  and  Swope  (1975)  for  the  Training  Effectiveness  Cost 
Effectiveness  (TECEP)  methodology.  They  started  by  developing  an  11- 
category  taxonomy  of  learning  outcomes  similar  to  that  espoused  by  Gagn§. 

Each  outcome  was  then  associated  with  a  set  of  prescribed  learning  activities. 
Some  learning  activities  are  common  to  all  types  of  tasks,  for  example,  inform 
student  of  the  objective,  provide  appropriate  practice  experiences,  and 
provide  performance  feedback.  Other  activities  are  more  appropriate  to 
certain  types  of  learning  outcomes.  Examples  of  specific  prescriptions 
include  (a)  acquisition  of  motor  skills  is  best  suited  to  hands-on  practice; 

(b)  visual  discrimination  requires  presentation  of  critical  stimuli;  and 

(c)  information  retention  is  benefitted  by  active  restatement  of  the 
information  in  different  contexts.  Included  in  the  prescription  for  each 
category  of  learning  outcome  is  the  specification  of  appropriate  training 
media  and  criteria  for  choosing  among  them.  The  media  selection  guidance  is 
provided  in  the  form  of  a  two-way  chart  indicating  whether  or  not  media  meet 
each  of  the  criteria.  The  user  indicates  which  of  the  criteria  must  be  met 
for  his/her  particular  application.  The  user  then  selects  the  media 
alternative(s)  that  most  closely  matches  his/her  set  of  criteria. 

Evaluation.  The  strength  of  the  device  selection  approach  typified  by 
TECEP  is  that  it  selects  devices  according  to  the  activities  required  to  learn 
the  tasks.  On  the  other  hand,  some  users  have  criticized  the  procedures  for 
classifying  learning  outcomes  and  for  specifying  learning  activities  as  not 
well  developed  (Vineberg  &  Joyner,  1980).  Other  users  have  suggested  that 
these  sorts  of  guidelines  are  primarily  aimed  at  the  training  developer  who  is 
conversant  with  instructional  technology  terms.  Those  who  are  not  conversant 
with  this  technology  are  likely  to  misunderstand  terms  that  are  essential  to 
the  application  of  this  model.  Another  criticism  of  the  TECEP  approach  is 
that  the  list  of  media  alternatives  is  overly  broad  with  a  bias  toward 
cognitive  (i.e.,  academic)  type  media.  Furthermore,  there  are  typically  no 
provisions  for  adding  to  or  modifying  the  list.  This  problem  is  particularly 
acute  for  gunnery  training,  which  is  supported  by  a  variety  of  special-purpose 


zThis  model  was  incorporated  in  the  Interservice  Procedures  for 
Instructional  Systems  Development  (Branson  et  al . ,  1975).  Therefore,  some 
researchers  identify  this  as  the  ISD  procedure  for  selecting  media. 
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training  devices  and  technologies.  Finally,  the  output  of  these  models  is 
limited  to  a  dichotomous  indication  of  whether  or  not  a  medium  is  appropriate 
for  training.  In  many  cases,  it  would  be  useful  for  the  training  developer  to 
have  more  scalar  information  about  the  devices,  for  example,  the  relative 
ranking  of  devices  with  respect  to  training  a  particular  objective. 

The  Automated  Instructional  Media  Selection  (AIMS)  Model 

Kribs  et  al.  (1983)  reviewed  extant  device  selection  models  and  provided 
criticisms  similar  to  those  stated  above.  They  also  argued  that  the  process 
of  matching  devices  to  training  requirements  is  needlessly  labor  intensive  and 
would  be  greatly  aided  by  computer  implementation. 

Methods.  Their  approach  to  device  selection,  the  Automated 
Instructional  Media  Selection  (AIMS)  model,  was  specifically  developed  in 
reaction  to  these  deficiencies  of  earlier  models.  In  general,  AIMS  takes  a 
less  theoretical  and  (therefore)  more  flexible  approach  to  device  selection. 

In  the  device  selection  approach  typified  by  TECEP,  both  device  and 
instructional  attributes  are  fixed.  In  contrast,  AIMS  users  generate  both  of 
these  sources  of  data  to  apply  to  their  particular  needs.  The  list  of  devices 
may  only  include  those  that  are  available  to  the  user  or  may  include  all 
applicable  devices,  available  or  not.  Instructional  attributes  may  include 
hardware/software  capabilities  (e.g. ,  existence  of  color  visual  displays)  or 
attributes  more  psychological  in  nature  (e.g.,  the  immediacy  of  feedback). 

The  instructional  attributes  provide  a  basis  for  describing  the  training 
requirements  and  device  capabilities.  The  AIMS  procedure  starts  with  the  user 
generating  a  list  of  media/devices  and  a  list  of  instructional  attributes.  He 
then  rates  the  degree  to  which  each  device  is  capable  of  delivering  each 
characteristic  on  a  scale  of  0  to  5.  A  rating  of  0  indicates  that  the  device 
has  essentially  no  capability  for  delivering  the  characteristic  in  question, 
whereas  a  rating  of  5  indicates  the  device  is  "very  capable"  of  delivering  the 
characteristic.  Next,  the  user  indicates,  for  each  task,  whether  or  not  each 
characteristic  is  required.  The  AIMS  software  then  calculates  the  average 
rating  and  rank  ordering  of  each  medium  over  all  attributes  ("general"  rating) 
and  the  average  rating  and  rank  ordering  for  only  those  attributes  that 
pertain  to  specific  tasks  ("specific"  ratings). 

Evaluation.  One  of  the  most  favorable  aspects  of  AIMS  is  its  inherent 
flexibility  and  adaptability.  Furthermore,  because  of  its  computer 
implementation,  the  model  is  able  to  perform  multiple  runs  of  the  data  under 
"what  if"  conditions.  On  the  negative  side,  the  ratings  data  are  not  handled 
in  a  mathematically  sophisticated  manner.  This  leads  to  two  specific  problems 
of  interpretation.  First,  the  averaging  procedure  assumes  that  the 
instructional  attributes  are  equally  weighted  over  all  objectives  as  well  as 
for  individual  objectives.  Consider  attributes  of  fidelity  that  could  be 
related  to  gunnery  such  as  the  capabilities  to  simulate  movement  and  to 
simulate  faring.  These  two  attributes  should  be  differentially  weighted  for 
tasks  such  as  "Conduct  a  Tactical  Formation"  versus  "Execute  a  Platoon  Fire 
Command."  Second,  AIMS  provides  not  one,  but  multiple  ratings  for  each  medium 
or  device.  As  in  so  many  instructional  development  procedures,  the  final 
decision  is  dependent  on  "expert  judgment."  To  come  to  a  less  equivocal 
selection,  the  process  must  somehow  aggregate  ratings  to  come  to  a  final 
decision.  To  aggregate  responses  (using  something  more  than  averages) 
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requires  some  theoretical  notions  of  the  mathematics  of  scaling  and  the 
psychology  of  decision  making. 

A  Model  Based  on  Multiattribute  Utility  Measurement  Technology 

An  alternative  to  the  TECEP  and  AIMS  models  of  media  selection  is  one 
based  on  multiattribute  utility  technology  (MAUT).  This  MAUT-based  model 
specifically  addresses  the  negative  aspect  of  AIMS  discussed  above.  That  is, 
MAUT  provides  methods  for  making  decisions  about  objects  (e.g.,  training 
devices)  having  value  on  a  number  of  different  dimensions  or  attributes.  The 
MAUT-based  media  selection  model  is  particularly  interesting  in  that  it  has 
both  heuristic  and  algorithmic  components.  The  initial  structuring  of  the 
decision  making  problem  is  usually  heuristic  in  nature.  In  that  regard, 
Edwards  (1987)  admitted  that  "...  structuring  is  an  art  form,  subject  to  few 
rules  and  little  prescription"  (p.  1066).  At  a  minimum,  this  heuristic 
process  should  produce  two  essential  products:  (a)  a  list  of  options  and  (b) 
the  value  structure  that  is  descriptive  of  the  consequences  of  each  option. 
Fortunately,  the  problem  of  media  selection  considerably  constrains  the 
problem  structure  in  two  ways.  First,  the  options  are  the  candidate  devices 
themselves.  Second,  the  value  structure  consists  of  attributes  that  would 
theoretically  influence  the  user's  choice  of  devices.  Devising  an  appropriate 
value  structure  is  clearly  more  difficult  than  determining  relevant  devices. 

The  algorithmic  component  of  the  method  includes  the  mathematical  and 
probabilistic  techniques  related  to  decision  making.  These  techniques  are 
algorithmic  in  that  they  ensure  that  a  unique  solution  (decision)  will  be 
reached  once  the  decision  maker  has  sufficiently  structured  the  problem. 
Furthermore,  these  mathematical  methods  are  transparent  to  the  user  and 
relatively  easy  to  implement.  More  sophisticated  models  of  MAUT  can  be  used 
to  address  the  media  selection  process,  but  Edwards  (1987)  argued  that  more 
elaborate  models  are  often  not  necessary  to  make  relatively  simple  decisions. 
He  referred  to  his  recommended  procedure  as  the  Simple  Multiattribute  Rating 
Technique  (SMART).  According  to  the  SMART  model,  the  overall  weighted  utility 
(U)  of  object  7  can  be  calculated  as 

U/  =  2  (w j  )(u fj)  (1) 

where  w j  =  normalized  importance  weight  of  dimension  j  (i.e.,  2  w j=  1.0), 
and  u fj=  utility  of  object  i  on  dimension  j 

Methods.  As  implied  in  the  previous  discussion,  the  most  difficult 
aspect  of  a  MAUT  analysis  is  the  establishment  of  the  value  structure.  The 
most  direct  definition  of  the  value  of  training  devices  would  be  in  terms  of 
the  training  effectiveness  related  to  each  training  device.  As  argued  in  the 
subsequent  section  (Allocate  Training  Time),  such  performance  data  are  very 
difficult  to  achieve.  The  more  realistic  alternative  is  to  rely  on  expert 
opinion.  The  following  paragraphs  present  two  possible  approaches  to  this 
problem. 

One  solution  to  the  value  structure  problem  is  to  define  subjective 
value  (i.e.,  utility)  in  terms  of  the  tasks  that  can  be  trained  on  a  device. 
For  instance,  Metzko  (1987)  used  a  MAUT-based  technique  to  evaluate  the 
utility  of  five  different  armor  gunnery  training  devices  for  use  in  the 
Reserve  Component:  (a)  the  Tank  Gunnery  and  Missile  Tracking  System  (TGMTS), 
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(b)  the  Mobile  Conduct-of-Fire  Trainer  (M-COFT),  (c)  the  Videodisc  Gunnery 
Simulator  (VIGS),  (d)  the  Tank  Weapons  Gunnery  Simulation  System  (TWGSS),  and 
(e)  Guardfist  1  (GF1).  Metzko  defined  the  utility  of  device  /  on  task  j  as  a 
simple  binary  rating  (yes/no)  indicating  whether  or  not  the  task  could  be 
trained  on  devices.  The  tasks,  in  turn,  were  rated  on  two  dimensions  of 
importance:  (a)  combat  criticality,  and  (b)  need  for  simulation.  These  data 
were  used  to  calculate  the  overall  utility  of  each  device.  The  utility  values 
were  then  combined  with  simulator  cost  data  to  calculate  the  cost-per-task- 
trained  for  each  of  the  five  devices.  These  cost-utility  analyses  were 
submitted  to  the  U.S.  Army  Armor  School  as  preliminary  evaluative  data  for 
decision  makers. 

Metzko' s  research  indicates  that  MAUT  can  be  used  to  make  practical 
decisions  about  gunnery  training  devices.  However,  there  are  problems  related 
to  defining  utility  in  terms  of  to-be-trained  tasks.  The  first  problem  is  the 
proliferation  of  tasks  that  relate  to  gunnery.  The  number  of  tasks  can  make 
the  analysis  tedious,  but  certainly  not  impossible.  The  second  and  more 
fundamental  problem  relates  to  answering  the  question:  Can  task  j  be  trained 
on  device  7'?  Hoffman  and  Morrison  (1988)  noted  in  their  rational  analysis  of 
gunnery  devices  that  answering  this  question  required  more  than  a  simple  yes 
or  no.  For  instance,  their  analysis  indicated  that,  for  many  tasks,  devices 
sometimes  simulated  only  parts  of  actions,  or  that  some  device  stimuli  and 
responses  varied  greatly  in  their  similarity  to  the  actual  equipment.  In 
other  words,  their  analysis  indicated  that  "tasks"  are  complex  concepts  and, 
as  such,  are  not  well  suited  to  serve  as  dimensions  upon  which  devices  should 
be  evaluated.  An  alternative  approach  is  to  rate  the  devices  on  attributes 
that  should  have  an  effect  on  learning.  Examples  of  device  attributes  include 
the  four  dimensions  of  fidelity  features  identified  by  Morrison  et  al.  (1990): 
(a)  interior  or  tank-appended  components,  (b)  exterior  visual  scene, 

(c)  movement,  and  (d)  system  failures.  Alternatively  (or  perhaps 
additionally),  other  device  attributes  could  be  considered  as  dimensions  of 
value,  such  as  the  instructional  features  or  performance  measurement  features 
of  devices.  In  order  to  use  these  attributes  in  a  MAUT  analysis,  each 
attribute  must  be  rated  in  importance  with  respect  to  particular  training 
applications  or  objectives.  Thus,  the  utility  of  a  particular  device  would 
vary  as  a  function  of  those  applications. 

Evaluation.  The  principal  advantage  of  the  MAUT-based  model  over  AIMS 
is  that  it  is  based  on  a  theoretical  model  of  decision  making  with  explicit 
mathematical  assumptions  that  allow  the  decision  maker  to  integrate  the  data 
in  a  meaningful  manner.  In  addition  to  this  increased  mathematical 
sophistication,  the  MAUT-based  model  also  retains  some  of  the  advantages 
inherent  in  the  AIMS  model.  First,  it  shares  with  AIMS  the  advantage  of 
letting  the  user  specify  the  objectives  to  be  trained  and  the  device 
evaluation  attributes.  Second,  MAUT  is  easily  computer  programmed  using 
spreadsheet  software  allowing  iterative  runs  of  the  data  under  "what  if" 
conditions.  The  obvious  next  step  in  development  of  this  method  is  to  try  it 
on  some  representative  training  problem. 

Summary  of  Recommendations  for  Selecting  Devices 

The  recommended  method  for  selecting  devices  is  the  MAUT-based  method 
described  above  and  illustrated  in  more  detail  in  the  following  section  on 
applications.  It  is  important  to  note  the  difference  between  previous 
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recommendations  (i.e.,  those  concerning  sequencing  and  structuring  training) 
and  the  present  recommendations.  The  previous  recommendations  were  stated  as 
general  guidelines,  that  is,  heuristics.  Assuming  the  training  developer  were 
reasonably  familiar  with  the  training  objectives,  he/she  could  use  the 
guidelines  to  arrive  at  appropriate  strategies  with  little  or  no  additional 
input  or  data.  In  contrast,  the  recommended  method  for  selecting  devices  for 
training  (the  MAUT-based  model)  is  based  on  "external"  sources  of  data,  that 
is,  information  from  sources  other  than  the  developer's  knowledge  of  the 
subject  matter  and  training  practices.  To  the  extent  that  these  data  can  be 
obtained  or  derived  from  objective  (i.e.,  performance)  sources,  the  analyst 
can  have  confidence  in  the  results.  However,  it  is  more  likely  that  the  data 
for  this  analysis  can  only  be  obtained  from  ratings  of  devices  and  training 
objectives  provided  by  subject  matter  experts.  The  analyst  must  therefore  be 
mindful  of  problems  related  to  expert  opinion,  and  take  steps  to  ensure  the 
reliability  and  validity  of  these  data. 

Allocate  Training  Time 

The  strategies  related  to  the  fourth  problem  (allocating  training  time) 
are  based  on  mathematical  algorithms  for  measuring  transfer  and  cost 
effectiveness.  These  methods  are  detailed  in  the  first  two  of  the  four 
sections  presented  below.  Like  the  methods  related  to  previously  described 
functions  (selecting  devices/media),  the  allocation  of  training  time  requires 
external  sources  of  data,  specifically  transfer  performance  data  and  detailed 
cost  information.  Collection  of  these  external  data  creates  special  problems 
that  impede  the  execution  of  these  methods.  Therefore,  data  collection  issues 
are  discussed  separately  in  the  final  two  sections.  Thus,  the  four  sections 
do  not  present  alternative  approaches  to  allocating  training  time;  rather, 
they  examine  different  aspects  of  the  same  approach. 

Measurement  of  Transfer  Effectiveness 


The  cost  effectiveness  measures  discussed  in  the  next  section  are  based 
on  fundamental  notions  of  transfer  effectiveness.  To  simplify  the  exposition, 
the  following  paragraphs  present  transfer  effectiveness  methods  without 
consideration  of  costs. 

Methods.  Transfer  of  training  measures  are  usually  derived  from 
performance  data  collected  from  a  transfer  experiment.  The  prototypical 
transfer  design  compares  the  performance  of  two  groups  of  subjects:  an 
experimental  group  who  receive  training  on  a  device  and  a  control  group  who  do 
not  receive  training  on  the  device.  Both  groups  are  then  tested  in  a  second 
medium.  The  second  medium  may  be  the  actual  equipment  itself  or  another 
(presumably  higher  fidelity)  training  device.  Superior  performance  of  the 
experimental  group  over  the  control  group  on  the  second  medium  is  taken  as 
evidence  of  transfer  of  training  from  the  first  to  the  second  medium.  Roscoe 
and  his  colleagues  (Roscoe,  1971;  Roscoe  &  Will iges ,  1980)  have  convincingly 
criticized  the  traditional  method  of  measuring  transfer  for  ignoring  the 
effects  of  practice.  They  cited  evidence  that  the  amount  of  transfer  is  a 
monotonically  increasing,  negatively  accelerated  function  of  the  amount  of 
practice  on  the  training  task.  In  other  words,  increasing  practice  on  the 
training  task  leads  to  decreasing  increments  of  criterion  performance.  Thus, 
transfer  effectiveness  is  not  a  single  value  but  a  function  relating  amount  of 
transfer  to  practice  on  the  training  task. 
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Roscoe  (Roscoe,  1971;  Roscoe  &  Williges,  1980)  discussed  several 
different  quantitative  methods  for  measuring  transfer,  arguing  in  favor  of  the 
cumulative  transfer  effectiveness  ratio  (CTER).  The  CTER  may  be  conceptually 
defined  as  the  training  time  saved  on  the  actual  equipment  due  to  training  on 
a  device  relative  to  the  time  spent  on  the  device  itself.  The  function 
relating  CTER  values  to  different  values  for  the  time  on  the  device  (the 
cumulative  transfer  effectiveness  function,  CTEF)  is  defined  as  follows: 

CTEF  =  ( Y0  -  Yx)/X  (2) 

where  Y0  =  time  or  trials  required  for  the  control  group  to  reach 
some  criterion  of  performance, 

Yx  =  time  or  trials  for  the  experimental  group  to  reach  the 
same  criterion  of  performance  after  X  hours  of  device 
training,  and 

X  =  time  or  trials  that  the  experimental  group  was 
pretrained  on  the  to-be-evaluated  training  device. 

Roscoe  (1971)  showed,  for  various  experimental  tasks,  that  the  CTEF  was 
a  decreasing  function  of  time  on  the  device,  approaching  zero  for  large  values 
of  X.  To  illustrate  the  calculation  of  CTER,  however,  Roscoe  used 
hypothetical  data  that  were  intended  to  represent  the  transfer  effects  of  a 
flight  simulator  on  actual  performance  in  a  commercial  aviation  aircraft.  His 
data  showed  decreasing  values  of  CTER  for  increasing  hours  on  the  simulator, 
indicating  that  the  transfer  effectiveness  of  the  simulator  declines  as  amount 
of  practice  increases.  When  the  function  decreases  to  a  value  of  1.0,  the 
ratio  indicates  that  the  training  time  savings  earned  by  training  on  the 
device  (Y0  -  Yx)  is  exactly  equal  to  time  on  the  device.  If  total  training 
time  were  the  only  factor  in  allocating  training  time,  the  training  device 
would  have  no  further  benefit  beyond  this  point.  That  is,  w're  time  would  be 
needed  on  the  device  than  on  the  real  equipment  for  the  same  effect  on 
learning. 

Holding  (1977)  proposed  a  simpler  and  more  useful  formulation  for 
measuring  transfer  effectiveness.  His  index  is  the  total  number  of  hours  or 
trials  associated  with  the  training  medium  and  the  transfer  device  required  to 
reach  some  performance  criterion  (or  X  +  Yx  in  Roscoe' s  terms).  This  index  is 
then  plotted  as  a  function  of  units  of  practice  on  X.  If  the  training  device 
is  effective  in  reducing  the  total  amount  of  training,  this  function  will 
decrease  to  a  point  and  then  increase  as  X  increases.  The  training  device  is 
assumed  no  longer  effective  (in  a  transfer  sense)  when  total  training  time 
(X  +  Yx)  exceeds  the  training  time  associated  with  the  transfer  device  alone 
(i.e.,  Y«).  Figure  1  compares  Holding's  transfer  index  with  the  CTER  for  the 
hypothetical  data  generated  by  Roscoe  (1971).  For  purposes  of  this 
comparison,  the  X  +  Yx  measure  is  expressed  as  a  proportion  of  the  total 
training  time  at  X  =  0,  or  Y«.  Thus,  the  total  time  function  starts  at  1.0, 
decreases  below  that  point,  then  increases  again.  The  point  at  which  the 
device  is  no  longer  effective  is  where  the  function  again  exceeds  1.0.  The 
figure  illustrates  that  both  the  CTER  and  Holding's  index  indicate  that,  for 
the  hypothetical  data,  this  point  occurs  at  about  five  hours  of  simulator 
training. 
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Figure  1.  Comparison  of  transfer  indexes  derived  by  Roscoe  (CTER)  and 
Holding  (total  hours).  (Hypothetical  data  from  Roscoe,  1971.) 


Implicit  in  both  Roscoe' s  and  Holding's  methods  is  the  strategy  to 
continue  training  on  the  device  to  the  point  at  which  the  device  no  longer 
saves  training  time.  As  clearly  suggested  by  Holding's  function,  however, 
there  is  another  strategy  that  might  be  to  used  allocate  training  on  the 
device  and  the  equipment.  That  is,  to  allocate  the  amount  of  training  on  the 
device  such  that  total  training  time  is  minimized.  This  would  be  an 
appropriate  strategy  for  allocating  training  given  constraints  on  training 
time.  As  can  be  seen  in  Figure  1,  that  point  occurs  between  two  and  three 
hours  of  training  on  the  device.  This  minimization  strategy  is  discussed  in 
more  detail  below  in  the  context  of  training  costs. 

Evaluation.  The  methods  for  measuring  transfer  effectiveness  are 
relatively  well  developed  mathematical  algorithms  and  appear  unambiguous  in 
their  application  to  gunnery  training.  One  problem,  however,  is  that  the 
methods  assume  that  for  each  objective  there  exists  a  single  performance 
criterion  to  which  crews  should  be  trained.  As  discussed  in  the  context  of 
standard  setting  (Hoffman,  Fotouhi,  Meade,  &  Blacksten,  1990),  performance  is 
not  well  described  by  a  single,  immutable  criterion  of  performance.  The 
implications  of  this  problem  for  the  present  methods  are  (a)  that  transfer 
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performance  may  be  measured  in  units  other  than  trials  (or  time)  to  criterion, 
and  (b)  there  may  be  multiple  indexes  of  performance. 

Measurement  of  Cost  Effectiveness 


If  the  cost  rates  (cost  per  unit  time)  for  training  with  a  training 
device  and  with  the  actual  equipment  media  were  equal,  one  possible  strategy 
to  allocating  training  is  to  allow  training  on  the  device  to  continue  until 
the  device  no  longer  saves  training  time,  that  is,  to  the  point  where  total 
training  time  on  the  device  and  actual  equipment  exceeds  time  to  train  using 
the  actual  equipment  alone.  Using  this  strategy  and  the  previous  data  in 
Figure  1  as  an  example,  one  would  allocate  five  hours  of  training  on  the 
device  before  transferring  to  actual  equipment.  However,  most  training 
devices  (e.g.,  computer-based  simulators)  are  designed  to  be  much  cheaper  than 
actual  equipment  in  providing  practice  on  to-be-learned  tasks.  Thus,  training 
devices  can  continue  to  be  cost  effective  beyond  this  point.  The  following 
paragraphs  discuss  methods  for  allocating  training  time  in  light  of  cost 
effectiveness  considerations. 

Methods.  If  cost  rates  were  constant  (i.e.,  training  costs  were  a 
simple  linear  function  of  training  time),  the  point  where  device  training  is 
no  longer  cost-effective  can  be  directly  calculated  from  the  transfer 
effectiveness  functions  discussed  above.  Using  the  CTEF  formula,  device 
training  is  no  longer  cost  effective  when  the  CTEF  crosses  the  point 
corresponding  to  the  ratio  of  cost/hr  on  the  device  to  cost/hr  on  the  actual 
equipment.  For  their  hypothetical  flight  data,  Roscoe  and  Williges  (1971) 
estimated  that  the  cost  rates  to  be  approximately  $16/hr  and  $24/hr  for  the 
simulator  and  the  actual  aircraft,  respectively,  resulting  in  a  ratio  of  about 
0.67.  Referring  back  to  Figure  1,  the  CTEF  crosses  that  ratio  at  about  11 
hours.  According  to  these  results,  then,  a  maximum  of  11  hours  of  simulator 
training  can  be  substituted  for  11  hours  of  flight  training  without  losses  in 
proficiency. 

Holding  (1977)  maintained  that  his  index  of  transfer  effectiveness  can 
be  used  to  integrate  cost  information  by  simply  converting  training  time  on 
the  media  to  costs.  Then  total  costs  (sum  of  costs  on  the  device  and  actual 
equipment)  are  plotted  as  a  function  of  device  training  time.  The  top  curve 
of  Figure  2  presents  this  total  cost  function  using  the  transfer  data  and 
estimated  cost  rates  provided  by  Roscoe  and  Williges  (1971),  that  is,  $16/hr 
and  $24/hr  for  the  simulator  and  the  actual  equipment,  respectively.  This 
function  can  be  interpreted  in  a  manner  similar  to  the  previous  total  time 
function  shown  in  Figure  1.  That  is,  training  with  the  device  is  no  longer 
cost  effective  when  total  costs  exceed  the  value  for  training  using  the  actual 
equipment  alone.  As  can  be  seen  in  Figure  2  and  in  agreement  with  the  CTEF 
analysis,  that  point  occurs  at  about  11  hours  of  training. 

The  total  cost  function  in  Figure  2  describes  all  possible  mixes  of 
training  on  the  device  and  actual  equipment  that  maintain  a  criterion  level  of 
performance.  This  function  illustrates  that  the  strategy  of  continuing  to  use 
a  device  until  no  longer  cost  effective  is  not  an  optimal  allocation  solution. 
That  is,  there  are  mixes  where  the  device  is  used  less  than  11  hours  and  the 
total  costs  are  less  than  the  costs  of  using  the  actual  equipment  alone. 
Bickley  (1980)  reasoned  that  the  optimal  allocation  of  training  on  a  device 
and  on  actual  equipment  is  that  point  where  total  costs  are  at  a  minimum.  To 
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Figure  2.  Components  of  the  total  cost  function.  (TOT  =  total  costs, 
TD  =  costs  associated  with  training  device,  AE  =  costs  associated  with 
actual  equipment.) 


develop  a  method  for  determining  the  minimum  point,  Bickley  first  decomposed 
the  total  cost  function  into  two  component  functions:  (a)  the  function 
relating  costs  associated  with  device  training  to  device  training  time,  and 
(b)  the  function  relating  costs  associated  with  training  on  actual  equipment 
as  a  function  of  device  training  time.  (These  two  components  are  shown  in 
relation  to  the  total  cost  function  in  Figure  2.)  The  first  relationship  is  a 
simple  linear  function  that  describes  the  cost  rate  of  the  training  device. 

The  second  relationship  is  sometimes  referred  to  as  an  "iso-performance" 
function  because  it  describes  the  tradeoff  between  device  and  actual  equipment 
costs  required  to  maintain  performance  at  some  given  level.  Bickley  showed 
that  this  iso-performance  is  satisfactorily  fit  by  a  decreasing  exponential 
function  of  the  form 

y  =  a  e'bx  +  c  (3) 

where  y  =  amount  of  training  in  actual  equipment  required  to  reach 
performance  criterion, 

x  =  amount  of  simulator  training,  and 
a,  b,  c  =  positive  constants  that  are  parameters  of  the  model. 
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Given  the  form  of  the  two  component  functions,  Bickley  was  able  to  derive  the 
formula  for  the  total  cost  function.  Using  simple  calculus,  he  then  derived  a 
formula  for  calculating  the  amount  of  device  training  that  minimizes  the  total 
costs.  Referring  to  Figure  2  again,  it  can  be  seen  that  that  point  is 
graphically  between  four  and  five  hours  of  training  on  the  device. 

Bickley' s  (1980)  approach  to  determining  the  optimal  allocation  of 
training  time  minimizes  costs  for  a  given  performance  level.  In  contrast, 
Carter  and  Troll  ip  (1980)  developed  methods  for  optimally  allocating  training 
time  such  that  performance  is  maximized  given  fixed  training  costs.  This 
solution  also  is  also  based  on  an  iso-performance  function;  however,  Carter 
and  Troll  ip  used  a  hyperbolic  form  that  was  much  simpler  than  the  exponential 
formula  assumed  by  Bickley.  The  user  must  specify  the  constraints  of  his/her 
budget  as  the  amount  of  dollars  available  for  training  and  the  relative  costs 
of  the  media.  For  instance,  suppose  $2000  were  available  for  gunnery  training 
with  training  on  actual  equipment  costing  $500/hr  and  training  on  a  computer- 
based  device  costing  $50/hr.  Expressing  these  constraints  as  an  equation, 

50A  +  500B  =  2000,  it  can  be  seen  that  different  sets  of  training  time 
allocations  can  satisfy  the  constraints  (e.g.,  0-4,  10-3,  20-2,  30-1,  40-0). 
Carter  and  Troll  ip  then  showed  that  the  Lagrange  multiplier  technique,  which 
was  developed  for  use  in  constrained  maximization  problems  in  microeconomic 
theory,  can  be  used  to  determine  that  unique  set  of  training  time  allocations 
that  yields  maximum  performance.  Despite  the  apparent  differences  between  the 
cost  minimization  approach  described  by  Bickley  and  the  performance 
maximization  approach  presented  by  Carter  and  Trollip,  Cronholm  (1985) 
demonstrated  that  the  two  approaches  resolve  to  identical  allocation 
solutions.  The  only  difference  between  the  optimal  solutions  is  the  criterion 
for  halting  training:  under  the  cost-minimization  solution,  training  on  the 
actual  equipment  is  halted  when  performance  on  the  actual  equipment  reaches 
the  performance  standard;  Under  the  performance  maximization  solution, 
training  is  halted  when  budgetary  resources  are  exhausted  (Sticha,  Singer, 
Blacksten,  Morrison,  &  Cross,  1988). 

Evaluation.  Three  different  methods  were  discussed  for  allocating 
training  time.  The  first  method,  implicit  in  the  analyses  described  by  Roscoe 
(1971)  and  Holding  (1977),  is  to  use  a  training  device  (or  any  lower  cost 
training  alternative)  until  it  is  no  longer  cost  effective.  The  second 
approach  (described  by  Bickley,  1980)  is  to  allocate  time  to  the  training 
device  to  achieve  a  fixed  standard  of  performance  such  that  total  training 
costs  are  minimized.  The  third  approach  (described  by  Carter  &  Trollip,  1980) 
is  to  allocate  training  time  given  a  fixed  total  training  budget  such  that 
performance  is  maximized.  The  analysis  by  Cronholm  (1985)  suggests  that  the 
two  latter  strategies  may,  in  fact,  yield  identical  allocation  solutions. 

There  is  no  preferred  approach:  The  relevance  of  the  three  methods  to  gunnery 
training  depends  on  the  intentions  of  the  training  strategy  designer. 

Collection  of  Transfer  Data 

The  techniques  for  measuring  transfer  are  conceptually  straightforward. 

It  is  telling  that  the  techniques  are  often  demonstrated  on  imaginary  data 
sets  such  as  the  hypothetical  data  generated  by  Roscoe  (1971).  The  major 
impediments  to  measuring  transfer  are  the  practical  problems  related  to 
collecting  transfer  data.  The  crux  of  this  problem  is  the  requirement  to 
measure  transfer  at  more  than  one  level  of  device  training.  The  following 
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paragraphs  describe  alternative  methods  that  have  been  suggested  for  obtaining 
such  data. 

Method.  If  level  of  training  is  manipulated  between  groups,  the 
required  number  of  subjects  becomes  a  crucial  problem.  Two  studies  that  have 
derived  transfer  functions  used  three  experimental  groups  of  subjects 
corresponding  to  different  levels  of  training  plus  a  fourth  no-training 
control  group  (Bickley,  1980;  Pouvenmire  &  Roscoe,  1973).  This  design  allows 
the  determination  of  four  points  on  the  transfer  function,  which  probably 
represents  the  minimal  requirement  for  estimating  such  functions.  On  the 
basis  of  a  power  analysis  of  gunnery  performance  measures,  Morrison  (1988) 
estimated  that  12  crews,  or  a  single  armor  company,  is  the  minimal  sample  size 
required  for  parametric  statistical  comparison  of  mean  differences.  Assuming 
four  samples,  this  implies  that  this  transfer  design  would  require  48  crews, 
or  all  the  crews  in  a  single  battalion.  For  determining  a  single  transfer 
function,  this  data  collection  requirement  may  be  unreasonable.  One  approach 
to  increasing  the  precision  of  the  estimate  between  training  time  and 
performance  (and  thereby  reduce  data  requirements)  is  to  obtain  initial 
performance  on  the  training  device.  If  this  performance  were  correlated  with 
performance  on  the  actual  equipment,  statistical  methods  could  be  used  to 
control  for  differences  among  crews  in  their  initial  (pretraining)  performance 
levels. 

The  between-groups  design  used  by  Pouvenmire  and  Roscoe  (1973)  and  by 
Bickley  (1980)  may  be  characterized  as  a  traditional  experimental  design  where 
equal  numbers  of  experimental  units  (here,  crews)  are  assigned  to  a  fixed 
number  of  conditions.  This  design  is  optimal  for  techniques  that  estimate 
mean  differences  (e.g.,  analysis  of  variance)  because  it  maximizes  statistical 
power  and  inoculates  the  results  against  violations  of  parametric  assumptions. 
However,  this  design  may  not  be  appropriate  for  nonlinear  regression  analyses 
whose  purpose  is  to  estimate  functions,  not  means.  To  estimate  such 
functions,  data  points  should  be  concentrated  about  the  critical  portions  of 
the  function  (e.g.,  inflection  points)  thereby  reducing  the  standard  error  at 
those  points.  For  total  time  or  cost  functions,  the  critical  portion  would  be 
around  the  minimum  point  of  the  function.  It  is  conceivable  that  this  design 
might  be  achieved  with  fewer  than  the  48  crews  prescribed  by  the  traditional 
design.  However,  much  more  information  is  needed  to  provide  sample  size 
requirements  for  the  optimal  design.  In  particular,  empirical  research  is 
needed  to  establish  the  minimum  number  of  crews  to  satisfactorily  fit  a 
learning  function,  and  the  number  of  data  points  required  to  accurately 
estimate  key  parameters  such  as  the  minimum  point  of  a  total  cost  function. 

Boldovici  (1987)  suggested  an  alternative  to  the  between-group  design 
wherein  amount  of  training  is  manipulated  within-groups.  Boldovici 's  design 
requires  an  experimental  group  to  alternate  between  training  and  testing  so 
that  they  are  repeatedly  tested,  at  different  points  in  learning.  A  second 
control  group  receives  only  the  multiple  test  trials  without  the  intervening 
training  trials.  The  repeated  measures  design  controls  for  repeated  testing, 
an  important  threat  to  internal  validity  of  the  design  (Campbell  &  Stanley, 
1966).  Internal  threats  to  validity  are  those  design  flaws  that  confound 
treatment  effects  with  other  effects  (e.g.,  testing  effects)  and  therefore 
seriously  compromise  the  interpretation  of  the  results.  By  including  a 
control  group  that  receives  the  test  trials  but  not  training,  Boldovici 's 
design  allows  the  researcher  to  control  for  and  evaluate  the  effects  of 
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testing  apart  from  the  effects  of  training.  That  is,  transfer  would  be 
measured  as  the  difference  between  the  experimental  group  and  control  group  at 
each  point  in  training.  Boldovici  also  pointed  out  that  the  multiple  test 
trials  in  the  control  group  allows  the  experimenter  to  calculate  the 
reliability  of  the  test  performance  measures. 

A  shortcoming  of  the  repeated  measures  design  is  that  it  does  not 
control  for  the  interaction  of  testing  with  training.  Campbell  and  Stanley 
(1966)  identify  this  interaction  as  a  potential  threat  to  external  validity. 

A  threat  to  external  validity  is  one  that  limits  the  general izability  of  the 
effects.  This  particular  external  threat  (interaction  of  training  and 
testing)  is  present  whenever  the  test  somehow  affects  or  sensitizes  subsequent 
treatments,  that  is,  the  interpolated  training  trial.  This  interaction  is 
quite  possible  in  the  present  transfer  design  where  training  and  testing  occur 
in  different  media.  In  fact,  one  of  the  sequencing  principles  (6.1)  discussed 
earlier  asserts  that  performance  is  enhanced  by  alternating  between  different 
training  devices.  In  other  words,  the  act  of  testing  on  some  high  fidelity 
medium  (e.g.,  the  actual  equipment)  may  enhance  the  effects  of  subsequent 
training  on  a  low  fidelity  alternative.  This  general izability  of  the  transfer 
effect  is  thereby  limited  to  those  situations  where  training  on  devices  and 
actual  equipment  is  alternated  exactly  as  prescribed  in  the  experimental 
design.  However,  practical  considerations  would  probably  proscribe  against 
this  iterative  approach.  The  more  traditional  approach  is  to  train  with  the 
lower-fidelity  alternative  and  then  shift  (once)  to  the  higher-fidelity 
medium.  Assuming  that  this  interaction  exists,  an  experimenter  would 
overestimate  the  transfer  between  devices  if  he/she  (mistakenly)  tried  to 
generalize  the  results  from  a  repeated  measures  design  to  the  traditional 
training  strategy. 

This  line  of  argument  points  out  that  the  "traditional"  approach  to 
training  may  not  be  the  optimal  strategy  in  that  it  does  not  allow  the 
transfer  potential  of  the  devices  to  be  fully  realized.  Thus,  the  interaction 
of  training  and  testing  may  be  viewed  as  a  training  strategy  as  well  as  a 
design  issue  in  transfer  of  training.  To  determine  the  extent  of  this 
interaction,  however,  would  require  the  addition  of  one  or  more  comparison 
conditions  wherein  training  and  testing  are  not  interspersed.  This 
alternative  approach  unfortunately  returns  us  to  the  problem  that  the  repeated 
measures  design  was  designed  to  address,  that  is,  the  excessive  numbers  of 
required  crews. 

Recognizing  the  severe  data  collection  requirements  for  determining 
transfer  functions  through  analysis  of  performance,  some  researchers  have 
turned  to  using  expert  judgments  as  proxy  measures  for  estimating  transfer. 
Pfeiffer  and  Horey  (1988)  refer  to  a  group  of  judgment-based  techniques  as 
"simulated  transfer."  To  use  this  technique,  personnel  familiar  with  the 
operational  task  in  question  and  the  training  device  are  asked  a  series  of 
structured  questions.  First,  they  are  asked  to  estimate  the  number  of  trials 
that  are  required  to  perform  a  particular  task  to  criterion  without  the  aid  of 
a  training  device.  Then  given  a  fixed  number  of  trials  on  the  simulator,  they 
are  asked  to  estimate  the  number  of  additional  trials  on  the  actual  equipment 
would  be  required  to  reach  the  same  criterion  after  device  training  has 
occurred.  From  these  data,  an  estimated  transfer  ratio  can  be  calculated.  To 
obtain  a  simulated  transfer  function,  the  expert  must  provide  multiple 
estimates  under  given  varying  amounts  of  device  training.  Unfortunately,  the 
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reliability  of  the  transfer  estimates  has  not  typically  been  measured  in  these 
studies.  Furthermore,  the  validity  issue  is  moot  given  the  lack  of  concurrent 
measures  of  actual  performance. 

Evaluation.  Neither  the  performance-based  nor  the  judgment-based 
approaches  to  determining  transfer  functions  is  completely  satisfactory.  The 
findings  from  the  empirical  performance-based  approach  (if  properly  executed) 
are  unequivocal;  however,  the  process  of  gathering  the  required  data  may  be 
prohibitively  expensive  in  terms  of  resources  required  to  obtain  and  test 
numerous  tank  crews.  On  the  other  hand,  the  resources  required  to  obtain 
expert  judgments  related  to  transfer  are  minimal;  however,  the  results 
obtained  may  have  questionable  validity.  There  ought  to  be  a  middle  ground 
wherein  knowledge  of  empirical  transfer  phenomena  (e.g.,  shape  of  the  transfer 
function;  cf.  Bickley,  1980)  can  be  used  to  improve  expert  judgment.  In  other 
words,  instead  of  obtaining  multiple  CTER  estimates,  judges  could  be  asked 
questions  relating  to  the  transfer  function's  parameters,  such  as  its 
asymptote  or  rate  parameter.  The  estimates  could  probably  be  made  relative  to 
other  devices  or  objectives.  The  estimates  could  then  be  used  to  generate  the 
functions.  This  approach  is  used  in  the  OSBATS-based  approach  described  by 
Blacksten  (1989).  Even  though  this  “enhanced  judgment"  approach  appears 
reasonable,  it  has  not  been  verified  by  comparison  with  actual  transfer  data 
or  with  other  simulated  transfer  techniques. 

Collection  of  Cost  Data 


Whereas  the  crux  of  the  problem  in  collecting  transfer  data  was  the 
required  soldier  support,  the  essence  of  the  problem  in  obtaining  cost 
information  itself  relates  to  the  nature  of  cost  data.  One  definition  of 
training  cost  is  the  value  of  important  opportunities  that  a  training  consumer 
must  forgo  in  order  to  avail  themselves  of  training  interventions  (Levin, 
1985).  With  reference  to  training  allocation  problems,  this  definition 
implies  that  the  cost  units  must  have  at  least  three  key  characteristics. 
First,  the  costs  should  be  relevant  in  that  they  measure  values  that  impact 
the  consumer  of  training.  Second,  cost  units  must  possess  certain  minimal 
mathematical  properties  (e.g.,  ratio  level  measurement,  commensurability  of 
units)  to  allow  the  units  reflect  value  in  a  mathematical  sense.  Third,  the 
cost  units  should  be  sensitive  enough  to  differentiate  among  the  training 
interventions.  Three  approaches  to  measuring  costs  are  discussed  below  in 
terms  of  these  key  characteristics. 

Methods.  Levin  (1985)  argued  that  total  costs  can  be  specified  by 
identifying  all  the  ingredients  of  a  training  intervention.  These  ingredients 
may  be  classified  under  general  headings  such  as  costs  related  to  personnel, 
facilities,  equipment,  supplies,  and  so  forth.  Once  all  ingredients  have  been 
identified,  the  analysis  proceeds  to  identify  who  incurs  each  cost.  This 
breakdown  allows  the  analyst  to  determine  the  "total"  costs  for  each 
constituency.  One  of  the  central  assumption  to  Levin's  ingredients  model  is 


30SBATS  is  an  acronym  that  stands  for  the  Optimization  of  Simulation- 
Based  Training  Systems.  Developed  by  Paul  Sticha  and  colleagues  for  the  Army 
Research  Institute,  OSBATS  is  a  computer-based  model  for  aiding  simulator 
developers  to  design  more  effective  training  devices.  For  more  detail  on 
OSBATS,  see  Sticha,  Blacksten,  et  al.  (1988). 
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that  the  best  estimate  of  the  value  of  each  ingredient  is  its  market  price. 
Techniques  are  suggested  for  estimating  the  market  price  for  each  of  the 
different  categories  of  ingredients.  This  technique  directly  addresses  the 
second  characteristic  of  costs  (the  mathematical  properties)  by  expressing 
values  in  a  common  physical  scale,  such  as  in  dollars  and  cents.  This 
approach  is  also  very  sensitive  to  differences  between  training  media.  On  the 
negative  side,  defining  costs  in  terms  of  dollars  and  cents  may  be  misleading 
in  some  instances. 

Edwards  (1987)  cautioned  decision  analysts  against  using  dollars  and 
cents  exclusively  in  measuring  perceived  value  (i.e.,  utility).  He  argued 
that  monetary  terms  may  be  appropriate  to  some  variables  but  not  others. 
Consider  the  example  of  training  from  the  viewpoint  of  a  military  unit 
commander.  Nonmonetary  costs  of  training  interventions  include  variables  such 
as  the  relative  availability  of  the  media  or  the  logistical  problems  of 
getting  to  and  from  training  sites.  He  might  also  consider  the  unit  personnel 
support  required  to  operate  the  media.  However,  the  personnel  costs  expressed 
in  purely  monetary  terms  would  not  be  appropriate  because  the  commander  is  not 
directly  accountable  for  these.  More  relevant  cost  measures  for  personnel 
support  include  the  degree  to  which  certain  types  of  personnel  can  be  replaced 
by  another  type.  For  costs  that  do  not  have  physical  scales,  the  alternative 
is  to  develop  subjective  scales  using  psychophysical  scaling  techniques.  The 
advantage  to  subjective  scales  is  that  they  can  be  tailored  to  correspond  to 
costs  that  are  relevant  to  the  user.  However,  subjective  scales  may  not  be 
commensurable  with  other  costs  on  physical  scales  of  measurement.  One 
possible  approach  for  combining  physical  and  subjective  measures  (even 
multiple  subjective  measures)  is  through  the  use  of  MAUT  techniques  discussed 
previously.  The  difference  is  that  previously  we  used  MAUT  to  combine  utility 
measures;  here,  MAUT  could  be  used  to  estimate  and  combine  cost  measures. 

The  Army  has  recently  focused  much  attention  on  a  single  cost  dimension 
that  may  prove  useful  in  solving  some  of  these  problems.  Operating  tempo 
(OPTEMPO)  is  presently  defined  in  terms  of  miles  per  unit  time  expended  by 
combat  vehicles.  By  making  unit  commanders  directly  accountable  for  this 
training  resource,  this  variable  is,  by  definition,  a  relevant  cost  factor. 
Another  advantage  to  OPTEMPO  is  that  actual  mileage  plus  certain  other 
automotive  expenditures  (e.g.,  maintenance)  get  converted  into  a  single 
metric,  miles.  The  only  problem  with  using  OPTEMPO  as  the  sole  cost  measure 
is  that  it  may  not  be  sensitive  to  certain  cost  differences  among  training 
media.  For  instance,  OPTEMPO  would  distinguish  between  tank-appended  devices 
that  consume  significant  OPTEMPO  resources  and  computer-based  devices  that 
consume  no  OPTEMPO.  However,  the  OPTEMPO  measure  would  not  differentiate 
among  alternatives  within  those  two  categories.  If  OPTEMPO  were  the  only 
significant  cost  factor,  then  this  insensitivity  would  not  pose  a  problem, 
that  is,  the  alternatives  within  categories  would  not  differ  in  costs. 

Evaluation.  Three  approaches  to  measuring  costs  were  discussed: 

(a)  determination  of  market  prices,  (b)  estimates  of  more  subjective  measures 
of  costs,  and  (c)  use  of  the  Army's  concept  of  OPTEMPO.  Neither  of  these  cost 
measures  was  entirely  satisfactory  for  gunnery  training  applications.  Perhaps 
the  best  approach  to  take  is  a  comprehensive  one  where  the  analysts  attempts 
to  identify  all  costing  ingredients  as  prescribed  in  the  method  proposed  by 
Levin  (1985).  In  contrast  to  Levin's  approach,  however,  the  ingredients 
should  not  consisting  exclusively  of  financial  variables.  The  problem  of 
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commensurability  may  be  addressed  by  transforming  all  variables  to  a  common 
subjective  utility  metric. 

Summary  of  Recommendations  for  A1 locating  Training  Time 

In  summary,  the  quantitative  methods  for  allocating  training  time  on  the 
basis  of  cost  effectiveness  are  relatively  unambiguous  applications  of 
mathematical  technology  and  present  no  serious  problems  of  interpretation. 
However,  the  use  of  these  methods  requires  valid  transfer  and  cost  data.  The 
collection  of  empirical  transfer  data  is  particularly  fraught  with  logistical 
and  design  problems.  One  solution  to  these  problems  is  to  derive  "simulated" 
transfer  functions  from  subject  matter  experts  who  are  partially  aided  by  our 
knowledge  of  the  functions  parameters.  However,  research  is  needed  to 
determine  the  extent  to  which  the  estimates  correspond  to  actual  performance. 
The  problem  with  cost  data  reduces  to  the  question:  What  is  a  cost? 
Alternative  definitions  were  offered,  each  with  advantages  and  disadvantages. 
The  recommended  solution  was  to  obtain  the  most  comprehensive  measures  of  cost 
and  examine  methods  for  combining  these  variables. 

Application  of  Training  Strategy  Methods 

In  the  previous  methods  section,  the  training  strategy  procedures  were 
described  in  the  abstract  so  that  they  have  some  generality  to  a  range  of 
problems.  The  purpose  of  the  present  applications  section  is  to  make  these 
methods  more  concrete  by  using  them  to  two  selected  aspects  of  the  domain  of 
training  objectives  identified  by  Morrison  et  al.  (1990).  These  two  cases 
were  chosen  to  be  sufficiently  different  from  one  another  to  demonstrate  the 
generality  of  the  methods.  The  aspects  were  also  chosen  to  illustrate 
prototypical  training  problems  that  confront  training  managers.  In  the  first 
case  presented  below,  the  methods  are  applied  to  the  problem  of  specifying  how 
devices  that  significantly  differ  in  fidelity  and  costs  can  be  used  to  train 
similar  skills.  In  the  second  case,  the  methods  are  used  to  determine  how 
devices  can  be  used  to  train  dissimilar  skills. 

As  previously  discussed,  the  heuristics  related  to  sequencing  and 
structuring  training  are  dependent  on  the  analyst's  knowledge  of  the  subject 
matter  (i.e.,  armor  gunnery)  and  of  training.  In  other  words,  the  application 
of  the  techniques  were  executed  without  relying  on  external  data  sources.  In 
contrast,  the  algorithms  related  to  device  selection  and  allocation  of 
training  time  required  external  data.  For  the  latter  two  groups  of  methods, 
hypothetical  data  were  generated  to  illustrate  the  application  of  the  methods. 
These  data  were  hypothetical  in  the  sense  that  they  were  not  obtained  from 
recommended  data  sources,  such  as  actual  performance  or  expert  opinion. 
However,  a  conscious  effort  was  made  to  approximate  the  data  that  would  be 
obtained  from  those  sources. 

Training  Similar  Skills  with  Different  Devices: 

Using  VIGS  and  I-COFT  to  Train  Target  Engagement  Skills 

For  many  domains  such  as  armor  gunnery  training,  both  high-  and  low- 
fidelity  training  devices  overlap  in  their  training  function:  that  is,  they 
train  a  common  set  of  skills.  The  issue  discussed  in  this  section  is  how  to 
integrate  training  so  that  both  types  of  devices  are  used  to  their  best 
advantage.  For  tank  gunnery  training,  two  different  computer-based  devices 
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have  been  prescribed  for  institutional  training:  the  Videodisc  Gunnery 
Simulation  (VIGS)  and  the  Institutional  Conduct-of-Fire  Trainers  (I-COFT). 

The  VIGS  is  an  inexpensive  tabletop  simulation  system  consisting  of  a  low 
fidelity  representation  of  the  gunner's  station,  a  laser  videodisc  player  for 
presenting  target  images,  and  a  micro-computer  for  controlling  the  simulation. 
VIGS  does  not  simulate  any  aspect  of  the  tank  commander's  station.  The  I-COFT 
is  a  special  version  of  the  core  simulator:  the  Unit  Conduct-of-Fire  Trainer 
(U-COFT).  In  training  capabilities,  the  I-COFT  is  very  similar  to  the  U-COFT 
in  that  they  are  both  high-fidelity  and  (relatively)  high-cost,  computer-based 
trainers.  In  contrast  to  the  VIGS,  the  I-COFT  has  the  capability  to  train 
tank  commanders  as  well  as  gunners. 

The  training  objectives  for  this  example  were  taken  from  the  cluster  of 
subtasks  related  to  target  engagement  identified  by  Morrison  et  al.  (1990). 
These  subtasks  were  similar  in  that  most  required  psychomotor  skills  and  a 
high  fidelity  representation  of  internal  tank  controls  and  external  visual 
scenes.  This  cluster  explicitly  excludes  subtasks  related  to  target 
acquisition.  For  the  present  example,  the  objectives  included  only  the 
subtasks  that  are  those  related  to  precision  (nondegraded)  mode  using  the  main 
gun.  Using  these  criteria,  nine  subtasks  were  identified  and  are  listed  in 
Table  4.  These  objectives  will  be  used  to  illustrate  methods  related  to  three 
of  the  four  problems  addressed  by  training  strategies:  structure,  sequence, 
and  allocate  training.  For  the  present  example,  the  device  selection  problem 
is  not  relevant  because  the  devices  (VIGS  and  I-COFT)  are  "givens"  in  the 
problem.  Despite  that  fact,  it  is  a  useful  first  step  in  developing  a 
training  strategy  to  systematically  evaluate  the  capabilities  of  both  devices. 

Evaluate  Device  Capabilities 

The  capabilities  of  VIGS  and  I-COFT  to  train  gunnery  engagement  subtasks 
are  summarized  in  the  last  two  columns  of  Table  4.  This  information  was 
adapted  from  a  detailed  evaluation  of  computer-based  gunnery  training  devices 
(Hoffman  &  Morrison,  1988;  Morrison  &  Hoffman,  1988).  The  devices  were 
evaluated  using  a  simple  three-category  scheme  that  describes  the  extent  to 
which  devices  represent  conditions  and  actions  associated  with  each  subtask: 
(a)  a  high  fidelity  representation,  (b)  a  low  fidelity  representation,  or 
(c)  no  representation.  As  can  be  seen  in  Table  4,  VIGS  is  a  lower  fidelity 
device  in  two  senses.  It  is  less  comprehensive  than  I-COFT  in  that  it 
provides  no  opportunity  for  training  certain  aspects  of  gunnery,  such  as  the 
TC  training  objectives.  For  the  subtasks  that  it  does  train,  it  is  also  less 
realistic  than  I-COFT.  For  instance,  not  all  the  switches  at  the  gunner's 
control  panel  are  represented  in  the  VIGS  as  they  are  in  the  I-COFT.  On  the 
basis  of  this  analysis  alone,  it  would  appear  that  the  I-COFT  is  the  preferred 
alternative  for  training  target  engagement  skills.  However,  the  purchase  cost 
of  VIGS  is  markedly  lower  than  I-COFT:  In  1988,  Witmer  estimated  that  VIGS 
sold  for  about  $40,000,  whereas  the  U-COFT  cost  nearly  50  times  that  amount  or 
$1,900,000.  (The  I-COFT  is  more  even  more  expensive  than  the  U-COFT  due  to 
its  enhanced  capabilities.)  As  a  consequence,  the  Army  can  purchase  more  of 
the  lower  cost  VIGS  devices  and  make  them  more  available  for  individual 
training.  Thus,  the  applied  research  question  should  not  be  "which  device  is 
better?"  Rather,  the  question  should  be  "how  can  these  devices  be  used  to 
their  best  advantage  in  an  integrated  training  strategy?"  This  question 
provides  the  central  theme  for  applying  the  training  strategy  methods. 
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Table  4 


Summary  of  To-Be-Trained  Subtasks  and  Device  Capabilities 


Subtask 

Crewman 

Performing 

Skill 

Fidelity  of 
Reoresentation 

VIGS  I-COFT 

3.1.1. 

Issue  standard  fire 
command 

TC 

Cognitive 

No 

High 

3.1.2. 

Lay  gun  for  direction 

i  TC 

Psychomotor 

No 

High 

4.1.1. 

Set  switches  per 
fire  command 

GNR 

Procedural 

Low 

High 

4.1.2. 

Identify  specified 
target 

GNR 

Visual 

perception 

High 

High 

4.1.3. 

Track  target 

GNR 

Psychomotor 

Low 

High 

4.1.4. 

Lase  to  target 

GNR 

Psychomotor 

Low 

High 

4.1.5. 

Fire  at  target 

GNR 

Psychomotor 

Low 

High 

4.2. 

Maneuver  tank 

DVR 

Psychomotor 

No 

No 

4.3. 

Load  round 

LDR 

Gross  motor 

No 

No 

Structure  Training 

The  subtasks  listed  in  Table  4  are  relatively  homogeneous  with  respect 
to  skills  and  fidelity  requirements.  On  the  other  hand,  the  numerous  subtasks 
in  this  cluster  represent  core  gunnery  skills  that  must  be  well  learned.  A 
key  question  in  this  problem  is  whether  this  cluster  should  be  considered  a 
single  instructional  unit,  or  subdivided  into  multiple  units.  On  the  basis  of 
the  complexity  of  the  target  engagement  task,  it  was  decided  to  break  the  unit 
down  further.  In  other  words,  a  part-task  strategy  was  to  prescribe  training 
on  this  gunnery  task.  Taking  this  as  a  working  assumption,  the  next  problem 
was  to  identify  the  parts.  Paraphrasing  the  previous  guidelines  in  the 
methods  section,  two  general  guidelines  for  partitioning  training  can  be 
derived:  (a)  sequentially  related  subtasks  can  be  trained  apart  if  they  are 
temporally  distinct  and  refer  to  different  subgoals,  or  (b)  nonsequential ly 
related  subtasks  can  be  trained  apart  if  the  underlying  skills  requirements 
are  substantially  different.  The  following  paragraphs  detail  how  these  two 
guidelines  apply  to  target  engagement. 

Two  subtasks  in  this  cluster  are  different  from  the  others  by  virtue  of 
the  fact  that  they  are  tank  commander  actions:  issue  fire  command  (3.1.1)  and 
lay  gun  for  direction  (3.1.2).  These  two  actions  occur  practically 
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simultaneously  raising  the  issue  of  whether  they  should  be  trained  separately 
or  together.  A  rational  analysis  of  skills  indicates  that  the  former  subtask 
is  primarily  cognitive  and  verbal  in  nature,  whereas  the  latter  is  primarily 
psychomotor.  According  to  Wicken's  (1989)  taxonomy,  these  two  subtasks 
require  substantially  different  attentional  resources.  Therefore,  they  can  be 
trained  apart  from  other  subtasks  in  this  cluster  and  apart  from  one  another. 

The  subtask  related  to  setting  switches  (4.1.1)  occurs  after  the  fire 
command  and  prior  to  tracking.  Furthermore,  the  skills  and  fidelity 
requirements  related  to  this  subtask  are  qualitatively  different  from  other 
subtasks  within  this  cluster.  Similarly,  the  subtask  related  to  target 
identification  (4.1.2)  is  a  temporally  distinct  action.  It  is  also  different 
in  a  skills  sense,  involving  perceptual  and  verbal  as  opposed  to  manual 
response  processes.  Thus,  both  of  these  subtasks  can  be  trained  separately 
from  the  others  in  this  cluster.  In  contrast,  the  track,  lase,  and  fire 
subtasks  (4.1.3,  4.1.4,  and  4.1.5)  are  described  as  interrelated  behaviors 
performed  in  sequence.  They  are  interrelated  by  virtue  of  the  fact  that  the 
behaviors  overlap  and  can  be  performed  out  of  sequence  under  some 
circumstances.  For  instance,  the  act  of  either  lasing  or  firing  can  cause  the 
gunner  to  lose  the  target  track,  requiring  him  to  restart  the  sequence.  In 
otherwords,  these  "sequential"  skills  are  less  sequentially  related  than  a 
simple  list  of  actions  would  indicate.  Furthermore,  all  three  subtasks 
require  similar  psychomotor  skills,  occur  in  close  temporal  proximity,  and  are 
related  to  a  common  goal  (i.e.,  hitting  the  target).  For  these  reasons,  these 
subtasks  should  be  trained  together  as  a  single  unit. 

Concurrent  with  the  actions  of  the  tank  commander  and  gunner,  the  driver 
and  loader  perform  related  subtasks:  maneuver  tank  (4.2)  and  load  round 
(4.3).  As  can  be  seen  in  Table  4,  these  objectives  are  not  trained  by  either 
simulator,  and  are  therefore  not  considered  in  the  present  example.  Thus, 
training  on  these  objectives  cannot  be  accomplished  on  these  two  simulators. 
Because  these  subtasks  are  no  longer  considered,  they  were  not  broken  down 
into  lower  level  subtasks  as  they  were  presented  in  Morrison  et  al.  (1990). 

In  summary,  the  analysis  indicated  that  the  su^tasks  related  to  target 
engagement  can  be  rationally  decomposed  into  parts  for  training.  At  the  same 
time,  the  parts  are  clearly  related  to  a  common  task  goal  (successful 
engagement  of  targets)  and  should  be  reintegrated  in  a  common  unit  of 
instruction.  In  training  development  terms,  the  common  training  unit 
addresses  some  terminal  training  objective  related  to  the  attainment  of  whole- 
task  gunnery  skills.  In  turn  this  objective  would  be  divided  into 
intermediate  training  objectives  corresponding  to  each  of  the  parts  identified 
above. 

Sequence  Training 

The  first  sequencing  consideration  is  how  the  unit  of  instruction  on 
target  engagement  relates  to  other  units  of  instruction  on  gunnery.  In  that 
regard,  the  present  unit  is  designed  to  train  prototypical  gunnery  skills, 
those  required  to  execute  the  simplest  gunnery  engagements  such  as  those 
having  only  a  single  main  gun  target  fired  under  nondegraded  mode  conditions. 
Accordingly,  this  unit  should  precede  other  units  that  concern  target 
engagements  under  special  circumstances,  such  as  multiple  targets,  machine  gun 
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engagements,  and  degraded  mode  gunnery.  This  sequence  is  prescribed  by  a 
number  of  rules  described  in  the  methods  section,  in  particular,  rule  1.3  (Use 
progressive  elaboration)  and  rule  4.2  (Progress  from  easy  to  difficult). 

The  actions  within  the  engagement  cluster  are  typically  performed  in  a 
fixed  order.  Therefore,  according  to  sequence  rule  2.1  (Follow  job 
performance  order),  the  subtasks  within  this  unit  should  be  sequenced  in  the 
order  that  they  are  performed.  However,  the  extent  to  which  this  rule  may  be 
carried  out  is  limited  by  device  capabilities.  That  is,  if  training  is  begun 
with  VIGS,  the  first  two  subtasks  related  to  the  fire  command  (3.1.1)  and 
initial  target  lay  (3.1.2)  cannot  be  trained.  One  way  to  alleviate  this 
problem  is  to  provide  off-line  (i.e.,  non-device-based)  verbal  instruction  on 
fire  commands  since  this  subtask  is  cognitive/verbal  in  nature  and  does  not 
require  a  simulator  for  training.  Off-line  training  on  fire  commands  could 
also  be  provided  in  conjunction  with  training  on  gunner  control  switches 

(4.1.1)  and  target  identification  (4.1.2).  More  problematic  is  the  subtask 
concerned  with  training  the  commander  to  provide  the  initial  target  lay 

(3.1.2) .  Because  this  psychomotor  skill  requires  simulation  of  a  target  and 
control  handles,  it  would  not  be  effectively  trained  by  verbal  methods. 

Because  the  VIGS  does  not  simulate  the  tank  commander  control  handle,  the  only 
alternative  is  to  train  this  subtask  on  the  I-COFT. 

At  this  point,  it  is  useful  to  summarize  the  results  from  the 
application  of  the  methods  related  to  both  structuring  and  sequencing 
training.  Figure  3  indicates  that  three  subunits  can  be  identified  for  this 
cluster  of  subtasks.  The  lowest  level  is  off-line  verbal  instruction  on  three 
subtasks  not  addressed  by  VIGS.  The  instruction  is  "off-line"  in  the.  sense 
that  instruction  is  not  in  the  context  of  one  or  the  other  training  device. 
Also  note  that,  for  these  relatively  independent  subtasks,  no  necessary 
instructional  sequence  is  implied.  The  next  level  of  instruction  is  provided 
by  VIGS,  wherein  the  student  receives  instruction  in  setting  switches  (4.1.1), 
identifying  targets  (4.1.2),  and  tracking/lasing/firing  (4. 1.3/4. 1.4/4. 1.5). 
These  three  groups  of  subtasks  should  be  trained  separately  and  in  sequence. 
After  initial  instruction,  however,  the  student  should  be  able  to  practice  all 
three  subtasks  as  an  integrated  act.  The  final  level  of  training  on  the 
I-COFT  integrates  tank  commander  actions  with  the  gunner  actions  presumably 
acquired  in  the  VIGS.  Note  that  instruction  at  this  level  should  emphasize 
the  initial  lay  (3.1.2)  because  this  is  the  first  opportunity  for  the  tank 
commander  to  practice  this  act. 

Given  the  structure  of  training  and  sequencing  within  the  three  subunits 
of  instruction  corresponding  to  the  three  media  (verbal  instruction,  VIGS,  and 
I-COFT),  the  next  step  is  to  specify  the  sequences  among  the  subunits.  Review 
of  the  sequencing  principles  reveal  two  different  strategies  could  be  taken. 
Perhaps  the  more  obvious  approach  is  to  follow  the  crawl-walk-run  principle 
(Rule  3):  (a)  Start  with  off-line  verbal  instruction,  (b)  proceed  to  training 

with  VIGS,  and  (c)  end  with  training  on  the  I-COFT.  The  other  approach  would 
follow  the  principle  of  alternating  devices  (Rule  6),  which  suggests  that 
alternating  practice  with  different  devices  may  enhance  learning.  Thus, 
alternating  between  VIGS  and  I-COFT  for  several  iterations  might  produce  more 
learning  than  simply  moving  from  the  lower-  to  the  higher-fidelity  device.  To 
determine  whether  the  alternation  sequence  is  superior  to  the  more  commonly 
accepted  notion,  research  should  be  performed  that  compares  these  two 
sequencing  strategies  while  strictly  controlling  amount  of  training.  However, 
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Figure  3.  Summary  of  structure  and  sequencing  strategy  for 
using  VIGS  and  I-COFT  to  train  gunnery  engagement. 


it  is  clear  that  the  former  strategy  (i.e. ,  train  on  VIGS,  then  transfer  to 
U-COFT)  would  probably  cause  fewer  logistic  problems  in  scheduling  training 
and  access  to  devices  than  alternating  back  and  forth  between  the  two  devices. 

Allocate  Training  Time 

For  the  purposes  of  illustrating  the  allocation  of  training  time,  the 
present  example  assumes  the  simpler  sequencing  strategy  detailed  in  the 
immediately  preceding  paragraph.  As  discussed  in  the  methods  section,  the 
allocation  of  training  time  requires  two  types  of  external  data:  transfer 
function  of  skills  between  VIGS  and  I-COFT  and  the  relative  costs  of  the  two 
devices.  The  transfer  data  requirements  are  discussed  in  the  paragraphs 
below,  followed  by  an  illustration  of  how  cost  information  can  be  integrated 
with  the  transfer  data  to  allocate  training  time. 

Two  research  studies  have  investigated  transfer  from  VIGS  to  U-COFT  with 
differing  results.  Witmer  (1988)  failed  to  demonstrate  reliable  transfer  from 
VIGS  to  U-COFT  or  from  U-COFT  to  VIGS.  On  the  other  hand,  Turnage  and  Bliss 
(1989)  demonstrated  that  skills  learned  on  low  fidelity  trainers  (VIGS  and 
another  device,  TopGun)  transfer  to  I-COFT.  However,  their  results  confounded 
the  effects  of  training  on  VIGS  with  the  effects  of  training  on  TopGun.  More 
importantly,  neither  Witmer  nor  Turnage  and  Bliss  systematically  varied  the 
amount  of  VIGS  pretraining  as  an  independent  variable.  Thus,  neither  study 
provided  data  sufficient  to  derive  a  transfer  function  of  skills  learned  on 
VIGS  to  I-COFT. 

Without  actual  transfer  data,  it  was  necessary  to  create  hypothetical 
data  to  illustrate  the  methods  related  to  this  problem.  These  data  would  be 
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obtained  from  the  following  experimental  design:  A  control  group  would 
receive  training  on  the  I-COFT  alone  while  at  least  three  experimental  groups 
would  receive  differing  amounts  of  pretraining  on  VIGS  prior  to  I-COFT 
training.  As  Bickley  (1980)  pointed  out,  the  determination  of  appropriate 
amounts  of  pretraining  for  the  experimental  groups  is  not  a  trivial  design 
detail: 

If  all  three  independent  variable  values  chosen  are  too  large,  then 
the  resulting  dependent  variable  values  will  all  lie  in  the 
asymptotic  portion  of  the  curve,  and  inferences  about  the 
descending  portion  of  the  [iso-performance  transfer]  curve  may  lack 
precision.  On  the  other  hand,  if  all  three  values  chosen  are  too 
small,  then  the  dependent  variable  values  will  all  lie  in  the 
descending  portion  of  the  curve,  and  inferences  about  the  magnitude 
of  the  asymptote  may  lack  precision  (pp.  9-10). 

Given  the  constraints  in  institutional  gunnery  training  and  some  knowledge  of 
the  effects  of  VIGS  training,  a  plausible  set  of  values  for  amount  of 
pretraining  for  three  experimental  groups  would  be  two,  four,  and  eight  hours. 
This  would  correspond  to  one-quarter,  one-half,  or  a  full  day's  practice  on 
VIGS.  These  hypothetical  data  represent  information  that  could  have  been 
obtained  from  the  results  of  a  transfer  experiment  or  from  estimates  of  those 
data  using  the  "simulated  transfer"  techniques  described  in  the  methods 
section.  The  present  data  were  generated  by  our  research  staff  who  were 
familiar  with  the  capabilities  of  the  two  devices  and  their  effects  on 
performance.  It  should  be  stressed  that  the  purpose  of  these  data  was  not  to 
provide  descriptions  of  the  transfer  relationships  that  actually  exist  between 
devices;  rather,  the  purpose  of  these  data  was  to  illustrate  how  such 
information  could  be  used  to  allocate  training  time. 

The  hypothetical  transfer  data  are  presented  in  the  top  row  of  Table  5. 
Data  from  the  control  group  (zero  hours  pretraining)  indicate  that  it  takes  an 
average  gunner  about  8  hours  (all  day)  to  learn  to  execute  some  sort  of 
engagement  or  a  related  set  of  engagements  to  some  criterion  of  acceptable 
performance.  Data  from  the  experimental  groups  indicate  that  the  time  to 
reach  criterion  on  I-COFT  is  inversely  related  to  increases  in  VIGS 
pretraining.  This  function  also  shows  the  negatively  accelerated  shape  that 
Roscoe  (1971)  noted  was  typical  of  skill  transfer.  The  second  row  in  the 
table  (total  time)  indicates  that  two  hours  of  VIGS  pretraining  effectively 
saves  about  one  hour  of  total  training  time.  Thus,  if  training  on  the  VIGS 
and  I-COFT  were  equally  costly,  the  optimal  mix  that  minimized  total  training 
time  would  be  two  hours  of  pretraining  on  VIGS  followed  by  (approximately) 
five  hours  of  training  on  I-COFT. 

As  argued  previously,  however,  the  costs  related  to  VIGS  and  I-COFT  are 
not  equal.  Witmer  (1988)  estimated  that  the  procurement  costs  of  U-COFT  are 
50  times  greater  than  those  of  VIGS.  It  was  also  noted  that  procurement  costs 
only  indirectly  affect  costs  incurred  by  the  unit.  The  effect  is  indirect  in 
that,  because  VIGS  is  cheaper,  it  can  be  made  more  available.  The  general 
fielding  strategy  calls  for  VIGS  to  be  made  available  at  the  company  level, 
whereas  some  version  of  COFT  will  be  provided  at  the  battalion  level.  A  very 
gross  estimate  of  the  relative  availability  of  VIGS  and  I-COFT  therefore  is  4 
to  1,  respectively,  based  on  the  relative  number  of  devices  dedicated  to  a 
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Table  5 


Findings  from  Hypothetical  Research  Measuring  Transfer  from  VIGS  to  I-COFT 


Hours 

on  VIGS 

0 

2 

4 

8 

Required  Hours  to  Reach 
Criterion  on  I-COFT 

8.0 

5.1 

4.3 

4.1 

Total  Time  (Hours) 

8.0 

7.1 

8.3 

12.1 

Total  Relative  Costs3 

32.0 

22.4 

21.2 

24.4 

aTotal  Relative  Costs  =  (I-COFT  hours  X  4)  +  VIGS  hours. 


battalion.  There  are  clearly  other  costs  related  to  using  VIGS  and  I-COFT 
such  as  the  support  required  and  logistics  related  to  getting  personnel  to  and 
from  training  facilities.  Nevertheless,  for  the  sake  of  simplicity  in  the 
present  example,  cost  rates  were  defined  exclusively  by  this  gross  estimate  of 
relative  availability  of  the  devices.  The  last  row  in  Table  5  displays  total 
costs  related  to  the  experimental  devices  using  these  relative  cost  estimates. 
The  results  show  that,  given  these  cost  estimates,  the  VIGS  continues  to  cost- 
effectively  substitute  for  I-COFT  at  even  eight  hours  of  VIGS  training.  This 
conclusion  is  based  on  the  fact  that  the  total  cost  of  training  at  that  point 
(24.4  "units")  is  still  less  than  the  cost  of  training  to  criterion  on  the 
I-COFT  alone  (32.0  "units").  However,  if  the  strategy  is  to  minimize  total 
costs  rather  than  substitute  VIGS  for  I-COFT,  a  VIGS/I-COFT  mix  of  (about) 
four  hours  on  the  VIGS  and  4.1  hours  on  I-COFT  appears  optimal. 

As  a  final  note,  this  example  assumes  that  performance  is  measured  using 
some  overall  composite  measure.  Although  there  may  be  some  merit  in  measuring 
overall  gunnery  performance  using  some  composite  measures  (cf.  Hoffman  & 
Witmer,  1989),  component  measures  of  performance  outcome  and  other  behavioral 
process  measures  provide  more  detailed  information  about  performance  on 
individual  subtasks.  (Suggested  measures  are  provided  in  Appendix  A  of 
Morrison  et  al . ,  1990).  However,  these  lower  level  measures  may  generate 
substantially  different  transfer  functions  that  the  overall  composite  measure. 
The  point  to  this  digression  is  that  the  solution  to  the  allocation  problem 
may  differ  depending  upon  the  exact  objective  (i.e.,  subtask)  that  is  being 
targeted  for  training. 

Training  Dissimilar  Skills  with  Dissimilar  Devices: 

Using  Various  Devices  to  Train  Platoon-Level  Gunnery  Skills 

As  in  the  previous  example,  the  training  objectives  are  defined  at  the 
subtask  level.  In  contrast  to  target  engagement  skills,  the  subtasks  related 
to  platoon-level  collective  training  are  heterogeneous  in  nature.  Table  6 
lists  these  subtasks,  which  were  identified  by  Morrison  et  al.  (1990). 
Furthermore,  a  variety  of  both  computer-based  and  tank-appended  devices  have 


Table  6 

Platoon  Collective  Subtasks 


1.  Travel  in  Platoon  Formation 

1.1.  Execute  a  Wedge  Formation 

1.2.  Execute  an  Echelon  Formation 

1.3.  Execute  a  Line  Formation 

1.4.  Execute  a  Vee  Formation 

1.5.  Execute  a  Column  or  Staggered  Column 

2.  Execute  Battle  Drills 

2.1.  Execute  Action  Drill 

2.2.  Execute  Contact  Drill 

2.3.  Execute  Air  Attack  Drill 

3.  Bound  by  Section 

4.  Overwatch  a  Bounding  Platoon 

5.  Occupy  a  Battle  Position 

5.1.  Occupy  Initial  Battle  Position 

5.2.  Occupy  Subsequent  Battle  Position 

6.  Maneuver  Within  a  Battle  Position 

7.  Employ  Fire  Patterns 

7.1.  Employ  Frontal  Fire 

7.2.  Employ  Cross  Fire 

7.3.  Employ  Depth  Fire 

8.  Employ  Firing  Techniques 

8.1.  Employ  Observed  Fires 

8.2.  Employ  Alternating  Fires 

8.3.  Employ  Simultaneous  Fires 


Note.  Taken  from  Morrison,  Meade,  and  Campbell  (1990). 


been  developed  to  train  these  objectives.  Thus,  this  example  is  good  for 
illustrating  the  methods  related  to  selecting  devices,  structuring  training, 
and  sequencing  training.  Although  the  methods  for  allocating  training  time 
are  not  well-suited  to  this  multi-task,  multi-device  example;  the  other 
remaining  three  groups  of  training  strategy  methods  can  be  applied  to  this 
second  example. 

Select  Devices 


As  concluded  in  the  methods  section,  the  MAUT  approach  is  the  method  of 
choice  for  selecting  training  devices.  The  MAUT-based  device  selection 
methodology  can  be  decomposed  into  four  steps:  (a)  formulate  an  initial  list 
of  training  devices,  (b)  select  attributes/describe  devices,  (c)  weight  the 
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attributes  according  to  training  objectives,  and  (d)  calculate  utilities  of 
the  training  devices.  Each  of  these  steps  is  described  by  example  below. 

Formulate  an  initial  list  of  trainino  devices.  The  first  step  of  this 
MAUT  analysis  is  to  identify  devices  that  may  apply  to  the  training  problem. 
With  regard  to  the  present  example,  five  different  training  devices  either  in 
current  use  or  in  product  development  were  identified  as  being  relevant  to 
training  platoon-level  collective  subtasks.  The  first  two  are  stand-alone, 
computer-based  training  devices,  whereas  the  remaining  three  are  tank-appended 
training  technologies.  Each  of  the  five  training  devices  is  briefly  described 
below. 

1.  Simulated  Networking  (SIMNET).  SIMNET  is  a  large-scale  research 
project  in  interactive  networked  simulation  sponsored  by  the 
Defense  Advanced  Research  Project  Agency  fr' 'VRPA)  and  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC).  Si.-iNET  provides  for  free 
play  scenarios  of  tactical  situations  on  a  computer-generated 
simulation  of  the  battlefield.  It  includes  simulator  stations  for 
a  number  of  different  types  of  fighting  vehicles  including  Ml 
tanks.  The  Ml  stations  consist  of  a  driver's  compartment  and 
station  and  a  crew  compartment  with  gunner,  loader,  and  commander 
stations.  The  stations  are  basically  low  fidelity  representations 
of  their  counterparts  on  the  actual  equipment,  and  do  not  allow 
crewmembers  to  practice  all  aspects  of  crew  gunnery.  For  instance, 
SIMNET  does  not  simulate  tank  machine  gun  effects  for  either  the 
coaxial  machine  gun  or  the  commander's  weapon.  For  more  details 
on  SIMNET  gunnery  training  capabilities,  see  Hoffman  and  Morrison 
(1988). 

2.  Platoon  Conduct  of  Fire  Trainer  (P-COFT).  P-COFT  is  a 
developmental  project  that  currently  exists  in  prototype  form  as 
the  UT-12.  P-COFT  extends  the  technology  developed  for  the  Unit- 
Conduct  of  Fire  Trainer  (U-COFT).  The  crucial  distinction  between 
the  devices  is  that  whereas  the  U-COFT  was  developed  for  training 
crew  gunnery  skills,  P-COFT  addresses  gunnery  skills  in  a  platoon 
context.  Like  U-COFT,  each  of  the  four  individual  P-COFT 
compartments  consists  of  gunner  and  tank  commander  stations  only. 

In  contrast  to  SIMNET,  however,  the  gunner  and  tank  commander 
stations  are  high  fidelity  representations  of  their  counterparts  on 
the  Ml  tank,  and  simulate  most  of  its  functions.  A  notable 
limitation  of  P-COFT  is  that  tactical  movement  is  strictly  limited. 
Like  the  U-COFT,  movement  occurs  at  constant  speed  along 
preprogrammed  routes;  tank  commanders  can  control  movement  only  to 
the  extent  that  they  can  start  and  stop  the  tanks.  The  computer¬ 
generated  terrain  for  the  prototype  UT-12  represents  a  typical 
Table  XII  range  (Range  301,  Grafenwbhr)  consisting  of  four  lanes 
with  artificial  defilade  positions.  Because  the  ultimate  design  of 
P-COFT  is  not  known,  the  attributes  of  the  UT-12  prototype  were 
used  to  evaluate  this  technology. 

3.  Multiple  Integrated  Laser  Engagement  Simulation  (MILES).  Appended 
to  actual  tanks,  the  MILES  system  simulates  weapons  effects  by 
transmitting  a  laser  beam  at  targets  equipped  with  MILES  sensors 
that  detect  target  hits.  The  principal  advantage  to  MILES  is  that 
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crews  use  their  own  tanks  to  practice  tactical  engagements  without 
the  safety  limitations  inherent  to  live-fire  ranges.  However,  the 
functional  similarity  of  MILES  to  live-fire  gunnery  is  limited  by 
the  fact  that  the  laser  beam  is  pointed  precisely  in  the  direction 
that  the  gun  tube  is  pointed.  In  an  actual  precision  engagement, 
the  ballistic  computer  modifies  the  gun's  aiming  point  in 
accordance  with  input  data,  for  example,  target  range,  ammunition, 
speed  of  target,  ammunition.  Therefore,  the  ballistic  computer 
must  be  turned  off  for  MILES  engagements.  This  deviation  implies 
gunner  does  not  set  switches,  lase,  and  lead  as  he  would  in  an 
actual  precision  gunnery  engagement.  Target  hit  effects  are 
simulated  by  the  flash  and  bang  emitted  from  Hoffman  devices  that 
are  appended  to  tanks.  However,  MILES  provides  no  simulation 
effects  for  target  misses.  Another  significant  departure  from 
fidelity  is  that  the  loader  is  not  required  to  load  actual  rounds. 

4.  Precision  Range  Integrated  Maneuver  Exercise  (PRIME).  PRIME  is  a 
developmental  project  whose  purpose  is  to  automate  a  number  of 
range  events  and  to  provide  performance  feedback  to  crews.  PRIME 
incorporates  a  number  of  different  technologies  including  an 
instrumented  version  of  MILES  (I-MILES).  I-MILES  identifies  firing 
vehicles,  allows  the  vulnerability  of  targets  to  be  varied,  and 
transmits  both  types  of  information  for  processing  and 
interpretation  in  terms  of  target  hits/kills.  In  addition  to 
processing  this  information,  PRIME  monitors  and  controls  other 
system  events  such  as  target  up/down  and  the  intervisibility  of 
targets.  The  targets  for  PRIME  are  radio  activated  silhouettes 
equipped  with  Laser  Target  Interface  Devices  (LTIDs) .  Thru-sight 
video  (TSV)  provides  the  capability  to  record  the  sight  pictures  as 
seen  through  the  gunner's  primary  sight  or  through  the  gunner's 
primary  sight  extension.  In  general,  PRIME  capabilities  mirror 
those  of  MILES  except  in  the  area  of  indirect  fires.  Whereas  MILES 
has  capabilities  for  representing  indirect  fires,  PRIME  does  not 
have  this  capability  in  its  present  configuration.  For  more 
details  on  the  capabilities  of  PRIME,  see  Drucker,  Campbell,  Koger, 
and  Kraemer  (1989). 

5.  Subcaliber.  The  tank-appended  Telfare  (M179)  subcaliber  device 
simulates  main  gun  effects  by  firing  a  Cal  .50  machine  gun 
coaxially  mounted  with  the  main  gun  at  appropriately  scaled 
targets.  One  of  the  principal  advantages  to  this  subcaliber  device 
is  that  weapon  effects  can  be  clearly  observed  and  used  for  aiming 
adjustments.  On  the  other  hand,  the  device  does  not  replicate  the 
obscuration  associated  with  the  main  gun  muzzle  blast. 

Furthermore,  the  subcaliber  round  is  slower  than  a  main  gun  round, 
thereby  making  it  unreal istically  easy  for  the  crew  to  visually 
follow  the  round.  As  with  live-fire  training  using  full-caliber 
ammunition,  subcaliber  firing  is  subject  to  range  fan  safety 
constraints. 

Select  attributes/describe  devices.  The  next  step  in  the  device 
selection  methodology  is  to  select  attributes  upon  which  the  devices  can  be 
evaluated.  For  this  example,  the  attributes  were  defined  as  characteristics 
of  a  device  that  determine  its  relevancy  for  training  certain  objectives. 
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There  are  many  possible  device  characteristics,  but  not  all  relate  to  training 
objectives.  For  instance,  the  capability  to  record  and  analyze  data  is  a 
characteristic  of  both  the  P-COFT  and  PRIME.  Even  though  this  attribute  may 
be  helpful  to  the  training  process,  the  training  objectives  do  not  differ  with 
respect  to  their  requirement  for  such  a  characteristic.  In  contrast,  the 
objectives  do  differ  with  respect  to  the  types  of  fidelity  required  to 
practice  the  task.  Hannaman  (1984)  identified  such  fidelity  features  for 
battle  simulations  in  general.  His  list  was  modified  to  capture  the 
similarities  and  differences  among  the  present  devices.  To  simplify  the 
analysis,  the  attribute  descriptions  were  defined  as  binary  characteristics 
(present/not  present).  Twelve  such  attributes  were  chosen  and  were  defined  in 
terms  of  the  following  yes/no  questions: 

1.  Defilade  position.  Can  the  simulated  tank  locate,  enter,  and  leave 
a  hull  defilade  and  turret  defilade  position? 

2.  Control  of  movement.  Can  the  crew  vary  the  speed  of  movement  and 
select  the  route  for  traversing  the  terrain? 

3.  Precision  gunnery.  Does  the  device  simulate  the  precision  fire 
control  system? 

4.  Degraded  gunnery.  Does  the  device  simulate  range  finder 
malfunctions  or  loss  of  stabilization? 

5.  Coaxial  machine  gun.  Does  the  device  simulate  the  M240  coaxial 
machine  gun? 

6.  Commander's  weapon.  Does  the  device  simulate  the  commander's 
caliber  .50  machine  gun? 

7.  Firing  signature.  Does  the  device  simulate  the  firing  signatures 
of  enemy  weapons  systems? 

8.  Weapons  effects  for  misses.  Given  a  miss,  does  the  device  provide 
a  realistic  cue  to  the  distance  between  the  point  of  impact  and  the 
target? 

9.  Indirect  fires.  Does  the  system  simulate  the  effects  of  indirect 
fires? 

10.  Company  net.  Does  the  device  simulate  communications  outside  the 
platoon  (i.e.,  on  the  company  command  radio  network)? 

11.  Obscurants.  Does  the  device  simulate  reduced  visibility  conditions 
caused  by  obscurants  such  as  smoke  or  fog? 

12.  Terrain.  Are  variations  in  terrain  represented  in  the  simulation? 

These  twelve  attributes  were  then  used  to  describe  the  fidelity  of  each 
of  the  five  devices.  The  results  are  presented  in  Table  7.  These  results 
showed  that  P-COFT  possessed  the  greatest  number  of  fidelity  attributes; 

SIMNET ,  the  least;  with  the  tank-appended  technologies  falling  somewhere  in 
between. 


Table  7 

Descriptions  of  Training  Devices  with  Respect  to  Fidelity  Attributes 


Attributes 

SIMNET 

Training  Devices 
P-COFT  MILES  PRIME 

SUBCAL 

1.  Defilade  Position 

0 

1 

1 

1 

1 

2.  Control  of  Movement 

1 

0 

1 

1 

i 

3.  Precision  Gunnery 

1 

1 

0 

0 

1 

4.  Degraded  Gunnery 

0 

1 

1 

1 

1 

5.  Coaxial  Machine  Gun 

0 

1 

1 

1 

0 

6.  Commander's  Weapon 

0 

1 

1 

1 

0 

7.  Firing  Signatures 

1 

1 

1 

1 

0 

8.  Weapons  Effects 

1 

1 

0 

0 

1 

9.  Indirect  Fires 

1 

1 

1 

0 

0 

10.  Company  Net 

1 

1 

1 

1 

1 

11.  Obscurants 

0 

1 

0 

0 

1 

12.  Terrain 

1 

1 

1 

1 

1 

Note.  One  (1)  indicates  that  the  device  possesses  the  attribute,  whereas  zero 
(0)  indicates  that  the  device  does  not. 


Weight  attributes  according  to  training  objectives.  Each  of  the  12 
attributes  was  weighted  with  respect  to  its  importance  for  training  the  19 
platoon-level  collective  subtasks  shown  in  Table  6.  A  ratio  weighting 
procedure  described  by  Edwards  (1987)  was  used  to  elicit  these  weights. 
According  to  this  procedure,  the  attributes  were  ordered  from  most  to  least 
important  and  then  the  least  important  attribute  was  assigned  a  weight  of  10. 
Each  succeeding  attribute  was  then  assigned  a  weight  by  asking  "how  many  times 
more  important"  was  the  attribute  from  the  least  important  one.  In  describing 
this  process,  Edwards  advised  that  the  analyst  should  not  strive  for  numerical 
precision;  rather,  he  should  strive  for  capturing  the  concept  of  importance. 
The  final  step  was  to  normalize  the  weights  by  making  them  sum  to  1.0.  This 
process  was  repeated  for  each  of  the  19  platoon-level  objectives.  The  results 
are  shown  in  Table  8. 
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Table  8 


Weights  of  Fidelity  Attribute  for  Platoon-Level  Subtasks 


Attributes* 


Subtasks 

1  2  3  4  5  6  7 

8 

9  10  11  12 

1.1 

WEDGE  FORM 

.014  .201  .115  .050  .101  .043  .029 

.094 

.022  .086  .072  .173 

1.2 

ECHELON  FORM 

.014  .201  .115  .050  .101  .043  .029 

.094 

.022  .086  .072  .173 

1.3 

LINE  FORM 

.015  .212  .106  .053  .121  .061  .038 

.030 

.076  .083  .023  .182 

1.4 

TRAVELING  OVERWATCH 

.070  .186  .109  .062  .078  .085  .124 

.047 

.016  .031  .039  .155 

1.5 

TRAVELING  COLUMN 

.015  .193  .119  .133  .037  .030  .059 

.052 

.044  .089  .067  .163 

2.1 

ACTION  DRILL 

.154  .185  .092  .054  .069  .046  .131 

.031 

.015  .062  .038  .123 

2.2 

CONTACT  DRILL 

.045  .195  .180  .090  .105  .053  .135 

.030 

.015  .075  .015  .060 

2.3 

AIR  ATTACK  DRILL 

.077  .231  .092  .015  .023  .123  .031 

.100 

.015  .046  .062  .185 

3.0 

BOUND  BY  SECTION 

.176  .200  .106  .094  .035  .047  .118 

.029 

.012  .018  .024  .141 

4.0 

OVERWATCH 

.157  .046  .118  .092  .052  .065  .131 

.039 

.013  .098  .020  .170 

5.1 

OCCUPY  INITIAL  BP 

.150  .165  .105  .090  .060  .068  .023 

.030 

.030  .053  .015  .211 

5.2 

OCCUPY  SUB  BP 

.150  .165  .105  .090  .030  .023  .015 

.045 

.038  .075  .053  .211 

6.0 

MANEUVER  IN  BP 

.159  .203  .087  .058  .022  .029  .101 

.065 

.014  .043  .072  .145 

7.1 

FRONTAL  FIRE 

.057  .043  .199  .184  .099  .092  .028 

.085 

.014  .021  .035  .142 

7.2 

CROSS  FIRE 

.057  .043  .199  .184  .099  .092  .028 

.085 

.014  .021  .035  .142 

7.3 

DEPTH  FIRE 

.057  .043  .199  .184  .099  .092  .028 

.085 

.014  .021  .035  .142 

8.1 

OBSERVED  FIRE 

.086  .043  .172  .155  .078  .069  .052 

.103 

.026  .017  .060  .138 

8.2 

ALTERNATING  FIRE 

.086  .052  .172  .155  .078  .069  .043 

.103 

.026  .017  .060  .138 

8.3 

SIMULTANEOUS  FIRE 

.100  .083  .183  .150  .133  .117  .042 

.067 

.017  .033  .025  .050 

•Attributes  are  identified  by  numbers  which  are  defined  In  the  previous  table  and  In  the  text. 


Calculate  utilities  of  the  training  devices.  Multiattribute  utilities 
were  calculated  for  each  of  the  training  objectives  (i.e.,  subtasks)  using  the 
SMART  formula.  The  results  are  shown  for  groups  of  related  subtasks  in 
Figures  4-7.  The  four  groupings  (travel,  position,  drills,  and  firing)  are 
based  upon  further  analysis  of  the  platoon-level  subtasks  identified  by 
Morrison  et  al.  (1990);  this  analysis  is  detailed  in  the  next  subsection. 


50 


Rttad  Otillty 


Figure  4.  MAUT  ratings  of  SIMNET  (SIM),  P-COFT  (P-C) .  MILES 
(MLS),  PRIME  (PRM) ,  and  Subcaliber  (SUB)  devices  for  training 
subtasks  related  to  travel. 
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Figure  5.  MAUT  ratings  of  SIMNET  (SIM).  P-COFT  (P-C),  MILES 
(MLS),  PRIME  (PRM),  and  Subcaliber  (SUB)  devices  for  training 
subtasks  related  to  positions. 
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Figure  6.  MAUT  ratings  of  SIMNET  (SIM),  P-COFT  (P-C),  MILES 
(MLS),  PRIME  (PRM) ,  and  Subcaliber  (SUB)  devices  for  training 
subtasks  related  to  drills. 
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Figure  7.  MAUT  ratings  of  SIMNET  (SIM),  P-COFT  (P-C),  MILES 
(MLS),  PRIME  (PRM),  and  Subcaliber  (SUB)  devices  for  training 
subtasks  related  to  firing. 
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The  findings  from  the  hypothetical  MAUT  analysis  may  be  summarized  by 
the  following  three  points: 

1.  In  general,  SIMNET  had  the  lowest  utility  of  the  five  devices  for 
training  platoon-level  collective  gunnery.  Out  of  that  domain, 
SIMNET  appears  best  suited  for  training  related  to  the  execution 
and  control  of  movement  formations.  (See  Figure  4.)  Nevertheless, 
even  for  these  subtasks,  the  rated  utility  of  SIMNET  was  exceeded 
by  that  of  both  the  Subcaliber  device  and  P-COFT.4 

2.  For  the  objectives  related  to  collective  firing  (Figure  7),  P-COFT 
had  the  highest  utility  of  all  five  devices.  This  is 
understandable  in  that  P-COFT  is  designed  to  train  such  "pure 
gunnery"  tasks.  In  contrast,  the  SIMNET  and  MILES  devices  were 
designed  primarily  to  train  tactics  with  gunnery  simulated  only  at 
a  rudimentary  level.  Had  PRIME  the  capability  to  simulate 
precision  gunnery,  it  would  have  compared  more  favorably  with 
P-COFT. 

3.  For  the  remaining  objectives  (Figures  5  and  6),  the  other  four 
devices  were  rated  fairly  close  in  utility.  Thus,  according  to 
this  analysis,  they  are  interchangeable.  The  reason  that  PRIME 
rated  slight  lower  than  MILES  is  its  inability  to  simulate  indirect 
fires. 

The  reasonableness  of  the  results  supports  the  contention  that  the  MAUT- 
based  methods  were  generally  useful  for  systematically  selecting  training 
devices.  The  results  also  point  to  two  specific  difficulties.  First, 
defining  the  utility  of  training  devices  in  terms  of  fidelity  features  may  be 
too  constraining.  This  definition  failed  to  capture  the  utility  of  certain 
instructional  features  such  as  the  performance  feedback  and  after  action 
review  capabilities  inherent  in  SIMNET  and  PRIME.  The  problem  in  including 
instructional  features  with  fidelity  features  is  that  it  obscures  the 
unidimensional  definition  of  utility.  One  solution  would  be  to  perform  two 
separate  analyses:  one  that  defines  utility  in  terms  of  fidelity  features, 
and  the  other,  in  terms  of  instructional  features.  The  attributes  for  the 


4The  fact  that  P-COFT  was  rated  higher  for  movement  subtasks  would  appear 
anomalous  because  it  does  not  allow  full  control  of  movement.  Note  that 
control  of  movement  was  indeed  the  most  important  attribute  for  this  set  of 
subtasks.  However,  this  positive  aspect  of  SIMNET  was  not  able  to  overcome 
its  deficiencies  with  respect  to  other  attributes  that  were  judged  important 
to  movement:  of  particular  note  is  SIMNET's  failure  to  simulate  defilade 
positions,  degraded  gunnery,  machine  guns,  and  battlefield  obscurants.  Each 
of  the  latter  features  was  supported  by  P-COFT.  Finally,  it  should  be  noted 
that,  because  the  simulated  tanks  in  P-COFT  move  independently  along  parallel 
tracks,  the  movement  techniques  and  some  formations  (e.g.,  wedge  and  line)  can 
be  executed  by  controlling  the  relative  speeds  of  the  tanks. 

5PRIME  originally  was  designed  to  be  equipped  with  the  Tank  Weapons 
Gunnery  Simulation  System  (TWGSS),  a  precision  gunnery  simulator.  Delays  in 
development  of  TWGSS  prevented  its  being  implemented  in  the  present  version  of 
MILES. 
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latter  analysis  would  be  the  individual  features  themselves.  There  exist 
several  inventories  of  instructional  features  (e.g.,  Sticha,  Singer,  et  al., 
1988)  that  could  be  used  to  generate  appropriate  attributes  for  the  analysis. 
The  analyst  could  use  results  from  both  analyses  to  reach  a  final  conclusion. 
The  problem  would  occur  when  the  fidelity-based  and  instructional  features- 
based  analyses  come  to  substantially  different  conclusions.  In  that  case,  the 
user  could  treat  the  two  utilities  as  attributes  in  another  MAUT  analysis, 
weight  them  accordingly,  and  reach  a  final  solution.  The  central  point  to 
this  discussion  is  that  instructional  features  of  devices  can  be  treated 
exactly  like  fidelity  features  in  a  MAUT-based  analysis  to  select  devices. 

The  second  difficulty  with  the  MAUT-based  model  was  that  the  analysis 
only  examines  utilities  while  ignoring  costs.  On  a  cost  basis,  the  computer- 
based  alternatives  (SIMNET  and  P-COFT)  would  clearly  be  preferred  because  they 
incur  no  OPTEMPO  resources  as  do  the  tank-appended  devices.  However,  as 
discussed  in  the  methods  section,  there  are  cost  elements  other  than  OPTEMPO. 
Assuming  that  device  costs  could  be  comprehensively  identified,  cost-utility 
analyses  (Levin,  1985)  could  be  performed  to  compare  device  selections  on  the 
basis  of  costs,  utility,  or  with  both  factors  considered.  However,  a 
systematic  consideration  of  costs  would  clearly  complicate  the  device 
selection  procedure  considerably. 

Structure/Seouence  Instruction 

There  is  little  systematic  precedent  for  the  application  of  sequencing 
principles  to  military  training,  and  there  exists  virtually  no  empirical 
literature  comparing  the  outcomes  of  different  training  sequences. 
Nevertheless,  it  is  instructive  to  illustrate  the  use  of  sequencing  principles 
derived  from  the  general  literature  of  the  psychology  of  learning  as  reviewed 
in  the  methods  section.  As  a  first  approximation  to  systematic  sequencing, 
further  simplifications  are  proposed  in  order  to  illustrate  the  application  of 
these  sequencing  principles  to  a  representative  set  of  armor  activities. 

Several  of  the  sequencing  principles  summarized  in  Table  3  can  be 
eliminated  without  serious  loss,  for  the  following  reasons.  First,  a  number 
of  the  principles  derived  from  differing  contexts  have  essentially  the  same 
import,  and  some  have  no  application  in  the  present  context.  Thus  rules  2.1, 
2.2,  and  2.3  (comprising  the  Natural  order  category)--while  distinguishable  in 
a  fine  analysis--convey  a  message  similar  to  5.2  (Natural  sequence). 

Principle  5.2  is  also  approximated  by  1.2  (Subject  logic)  at  some  levels,  but 
the  distinction  will  be  preserved  for  application  to  subtasks.  Again,  item 
3.3  (Practice  different  modes)  will  have  almost  the  same  impact  upon  on  armor 
training  as  6.1  (Vary  practice).  With  respect  to  item  4.1  (Transfer  order), 
too  little  information  is  available  on  transfer  of  tracing  to  allow 
deductions  to  be  drawn.  Another  item,  5.1  (Overlapping  practice)  may  be  ruled 
out  of  consideration  on  the  grounds  that  the  principle  addresses  the 
sequencing  problem  at  a  different  level,  affecting  the  integration  of  units 
rather  than  the  order  in  which  the  units  are  presented  for  training.  As  an 
outcome  of  these  considerations,  it  is  possible  to  provide  the  following 
residual  list  of  nine  principles: 
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1.1  Teach  ends  before  means. 

1.2  Follow  the  logic  of  the  subject  matter. 

1.3  Use  progressive  elaboration. 

3.1  Move  from  slow  to  fast  practice. 

3.2  Move  from  orientation,  through  practice,  to  real  task. 

4.2  In  general,  progress  from  easy  to  difficult. 

5.2  Follow  the  natural  sequence  of  the  task. 

6.1  Vary  practice  between  training  devices. 

6.2  Provide  real  equipment  experience  early  in  training. 

It  must  next  be  decided  at  what  levels  the  sequencing  principles  should 
be  put  into  practice.  To  illustrate  the  application  of  tnese  rules,  it  was 
useful  to  first  structure  the  list  of  platoon  collective  training  objectives 
as  an  organized  hierarchy.  The  analysis  by  Morrison  et  al.  (1990)  suggested 
that  the  19  subtasks  were  best  grouped  into  the  eight  task  groupings. 

However,  as  shown  in  Figure  8,  further  analysis  showed  that  certain  clusters 
may  be  combined  to  form  superordinate  units.  Thus,  although  group  1  (Travel 
in  platoon  formation)  and  group  2  (Execute  battle  drills)  were  already 
sufficiently  comprehensive,  groups  3  (Bound),  4  (Overwatch),  5  (Occupy),  and  6 
(Maneuver)  could  all  be  considered  instances  of  the  more  general  category 
concerned  with  "Field  position."  Again,  group  7  (Fire  patterns)  and  group  8 
(Firing  techniques)  could  both  be  treated  as  variants  of  a  general  "Firing" 
category.  Furthermore,  it  appeared  that  the  four  resulting  categories  might 
be  grouped  into  two  broad  classifications.  The  combination  of  group  1 
(Travel)  together  with  the  new  category  of  "Field  position"  could  be  regarded 
as  a  broad  category  representing  "Movement;"  the  combination  of  group  2 
(Battle  drills)  with  the  new  category  "Firing"  constituted  the  broad  category 
"Engagement".  Hence,  it  was  possible  to  depict  the  platoon  subtasks  in  the 
form  of  the  hierarchy  as  shown  in  Figure  8. 


Level 
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Tasks 
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Figure  8.  Hierarchical  arrangement  of  platoon  collective  subtasks. 
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Sequence  higher  level  elements  (categories  and  clusters).  With 
reference  to  Figure  8,  the  sequencing  problem  was  considered  at  each  of  the 
levels  A,  B,  C,  and  D.  However,  it  appears  that  few  of  the  sequencing 
principles  were  readily  applied  to  the  upper  levels  of  the  hierarchy.  At 
level  A,  the  only  question  was  whether  Movement  (M)  should  precede  or  follow 
Engagement  (E).  Since  Engagement  could  be  considered  the  end  in  relation  to 
Movement  as  the  means,  principle  1.1  (Ends  before  means)  apparently 
recommended  the  order  E  ->  M.  Principle  1.2  (Subject  matter  logic)  appeared 
only  applicable  to  sequencing  within  units,  as  was  largely  true  of  several 
other  principles  (1.3.  Progressive  elaboration;  3.1.  Slow-to-fast;  3.2. 
Orientation-practice-real).  However,  given  the  assumption  that  Movement  is  a 
less  difficult  task  than  Engagement,  application  of  principle  4.2  (Easy-to- 
difficult)  yielded  the  order  M  ->  E;  the  same  order  was  implied  by  principle 

5.2  (Follow  natural  sequence),  assuming  that  Movement  normally  precedes 
Engagement.  At  this  level  (A),  principle  6.1  (Vary  devices)  seemed  equally 
satisfied  by  the  orders  M  ->  E  or  E  ->  M. 

Before  considering  the  application  of  principle  6.2  (Real  equipment 
experience),  it  was  necessary  to  recall  the  results  of  the  device  utility 
ratings  listed  earlier.  In  general,  the  ratings  suggested  that  the  principal 
devices  were  almost  equally  valuable  for  tasks  1  through  6,  although  SIMNET  is 
somewhat  less  useful  on  many  subtasks.  However,  P-COFT  was  notably  superior 
for  task  7  (Fire  patterns),  with  Subcaliber  as  next  most  suitable;  these 
rankings  remained  very  similar  for  task  8  (Firing  techniques).  In  assigning 
devices  to  training  units,  it  was  assumed  that  the  principal  devices  for 
training  in  tasks  7  and  8  is  the  P-COFT  and  Subcaliber  devices,  with  greater 
flexibility  available  in  tasks  1-6.  However,  with  respect  to  principle  6.2 
(Real  equipment),  it  should  be  remembered  that  there  may  be  training  utility 
in  tank  exercises  not  connected  with  any  device,  which  might  be  inserted  into 
the  earlier  tasks  (1-6).  The  simplest  interpretation  might  be  that  category 
E,  which  contains  tank  practice  in  the  form  of  Subcaliber,  should  precede 
category  M  training.  However,  the  Subcaliber  device  was  marginally  the  most 
effective  in  most  of  the  Movement  tasks,  so  that  there  was  insufficient 
evidence  to  warrant  reversing  the  M  ->  E  order. 

The  sequencing  procedure  appeared  somewhat  inconclusive  at  this  level, 
but  the  results  became  clearer  with  further  progress  down  the  hierarchy.  The 
outcomes  of  applying  the  selected  sequencing  principles  at  level  A  are  listed 
in  Table  9,  as  are  those  for  the  following  level  (B).  As  an  example  of  the 
arguments  at  this  intermediate  level,  consider  the  implications  of  principle 

4.2  (Easy-to-diff icult).  The  decision  at  level  A  to  place  M  before  E  implied 
that  Travel  (T)  and  Position  (P)  clusters  should  be  less  difficult  than  the 
Drills  (D)  and  Firing  (F)  clusters.  Within  each  grouping,  T  appeared  simpler 
than  P,  and  D  appeared  simpler  than  F.  Hence,  the  order  T  ->  P  ->  D  ->  F  was 
recorded  in  Table  9.  Other  decisions  followed  similar  lines,  as  shown  in  the 
body  of  the  table.  The  consensus  at  the  second  level  (B)  appeared  to  favor  the 
partial  order  T  ->  P  ->  D,  although  it  was  unclear  whether  F  should  have 
preceded  or  followed  the  other  components.  As  a  temporary  expedient,  it  was 
assumed  that  the  order  T  ->  P  ->  D  ->  F  is  primary,  with  F  remaining  in  last 
place.  The  problem  raised  by  principle  6.2  (real  equipment),  which 
tentatively  placed  cluster  F  in  first  place,  was  handled  by  ensuring  that  the 
devices  recommended  for  training  cluster  T  (Travel)  occur  first  and  include 
real  tank  experience  in  some  form  (perhaps  using  the  Subcaliber  device,  as 
noted  above).  Note  that  the  deferment  of  firing  practice,  a  basic  skill,  was 
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only  possible  here  on  the  assumption  that  platoon  training  had  been  preceded 
by  earlier  gunnery  practice.  A  further  complication  was  that  principle  1.1 
(Ends  before  means)  appeared  to  be  at  odds  with  all  of  the  later  principles. 

A  compromise  solution  was  to  drop  1.1  from  the  main  analysis,  but  to  hold  the 
principle  in  reserve  for  use  as  necessary  to  resolve  ties  between  the 
remaining  set  of  principles. 


Table  9 


Application  of  Sequencing  Principles  at  Higher  Levels  of  the  Hierarchy 


Level 

Order 

Principle 

1.1 

1.2  1.3  3.1  3.2  4.2 

5.2  6.1 

6.2 

A 

1 

E 

M 

M 

M 

2 

M 

E 

E 

E 

B 

1 

F 

T 

T 

F 

2 

D 

T  P 

P 

T 

3 

P 

P  D 

D 

P 

4 

T 

F 

F 

D 

Note.  E  *  engagement;  M  *  movement;  D  =  drills;  F  =  firing;  P  =  positioning; 
T=travel.  A  dash  (-)  indicates  that  the  principle  has  no  apparent 
application. 


Sequence  lower  level  elements  (tasks  and  subtasks).  Decisions  were  then 
made  at  the  next  lower  level  (C).  where  the  eight  main  task  types  had  to  be 
ordered.  The  ordering  was  first  made  on  the  basis  of  the  Travel  -> 

Position  ->  Drills  ->  Firing  sequence  adopted  at  the  higher  level,  but  was 
then  reconsidered  regardless  of  the  prior  order  to  find  whether  the  procedure 
has  introduced  any  anomalies.  The  starting  order  of  tasks  within  clusters 
therefore  became:  (T)  1,  (P)  3  4  5  6,  (D)  2,  (F)  7  8  . 

Next,  training  devices  were  provisionally  assigned  to  tasks  in  order  to 
evaluate  compliance  with  principles  6.1  (Vary  devices)  and  6.2  (Real 
equipment).  Reviewing  the  ratings  assigned  above  to  device  utility,  for 
example,  showed  that  task  1  was  best  served  by  the  Subcaliber  device  (S),  or 
else  by  P-COFT  (C)  or  MILES  (M).  After  device  C,  task  2  was  best  served  by  M 
or  by  PRIME  (P).  Table  10  shows  the  highest  ranking  devices  for  each  task, 
together  with  a  suggested  choice  based  on  making  some  use  of  all  the  Drincipal 
devices  (C,  M,  P,  S)  so  as  to  ensure  that  principle  6.1  (Vary  devices)  was 
satisfied.  Since  there  were  many  ties  among  the  utility  ratings,  many  other 
device  selections  would  have  been  almost  equally  acceptable.  It  was  assumed 
that  SIMNET  (N)  would  also  be  introduced  from  time  to  time  to  provide  variety 
of  practice. 
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Table  10 

Assignment  of  Devices  to  Tasks  in  Clusters 


Cluster 

Travel 

Position 

Drill 

Firina 

Task 

1 

3  4 

5  6 

2 

7  8 

Ranked  devices 

S 

M  C 

S  S 

C 

C  C 

C 

P  M 

C  C 

M 

S 

M 

P 

M 

P 

Assigned 

S 

P  C 

C  S 

M 

S  C 

Note.  Letters 

refer  to 

devices: 

C  =  P-COFT; 

M  =  MILES; 

P  =  PRIME; 

S  =  Subcaliber. 

Numbers 

denote  tasks  within 

clusters: 

1  =  travel; 

2  =  drills;  3  =  bound;  4  =  overwatch;  5  =  occupy;  6  =  maneuver; 
7  =  patterns;  8  =  techniques. 


The  starting  order  for  tasks  1  through  8,  suggested  by  the  above 
application  of  the  principles  of  sequencing  to  levels  A  and  B  of  the 
hierarchy,  was  summarized  as  laid  out  in  the  first  row  of  Table  11.  Next, 
each  principle  was  applied  in  succession  to  ordering  the  tasks  that  constitute 
level  C  of  the  hierarchy.  The  method  consisted  of  first  making  successive 
modifications  to  the  orders  of  tasks,  while  preserving  the  order  of  task 
clusters.  It  should  be  noted  that  no  algorithm  existed  for  deciding  the  order 
in  which  to  apply  the  sequencing  principles.  The  method  adopted  was  to 
consider  each  principle  in  the  order  originally  derived,  using  its  application 
to  make  any  appropriate  changes  to  the  immediately  prior  ordering.  Where  any 
principle  had  no  application,  the  order  was  simply  left  unchanged.  All  of  the 
resulting  modifications  are  listed  in  Table  11.  It  can  be  seen  that,  at  least 
in  this  instance,  very  few  changes  in  order  were  enjoined  by  the  successive 
application  of  the  different  sequencing  principles.  In  addition,  the  table 
shows  that  the  final  order  presented  an  acceptable  variety  of  devices  that 
included  early  practice  on  tank-mounted  devices.  The  most  difficult  practical 
decisions  were  occasioned  by  principles  3.1  (Slow-fast)  and  4.2  (Easy- 
difficult)  as  applied  to  tasks  2  (Drills),  7  (Fire  patterns),  and  8  (Firing 
techniques).  The  order  8  ->  7  ->  2  appeared  almost  equally  good,  but  this 
sequence  contradicted  the  decision  earlier  in  the  hierarchy  to  place  F  after 
D.  The  chosen  final  order,  as  represented  in  the  table,  therefore  reflected 
the  earlier  decision.  However,  the  problem  raised  the  question  whether  the 
same  final  sequence  would  be  recommended  if  the  higher-order  categorization 
were  to  be  disregarded. 

Although  the  decisions  regardless  of  clustering  at  the  higher  level  were 
made  independently,  it  was  inevitable  that  applying  the  same  considerations 
produces  similar  order.  Note,  however,  that  the  results  might  be  similar  but 
not  identical.  For  example,  although  cluster  P  as  a  whole  precedes  clusters  D 
and  F,  the  effect  might  have  been  due  to  the  influence  of  tasks  3  (Bound),  5 
(Occupy),  and  6  (Maneuver);  task  4  (Overwatch)  could  arguably  be  relocated  on 
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the  basis  of  principles  3.1  (Slow-fast)  or  4.2  (Easy-difficult).  In  fact, 
ha  ever,  constructing  a  new  table  by  applying  the  same  procedures  used  in 
Table  11  while  disregarding  the  cluster  structure  produced  identical  results 
for  the  platoon  collective  tasks.  It  was  therefore  superfluous  to  reproduce 
the  second  table. 


Table  11 

Task  Orders  Resulting  from  Successive  edifications  Within  Clusters 


Principle 

Tasks 

in  Order 

Starting  Order 

1 

3 

4 

5 

6  2 

7 

8 

1.2 

1 

3 

4 

5 

6  2 

8 

7 

1.3 

1 

4 

3 

5 

6  2 

8 

7 

3.1 

1 

4 

3 

5 

6  2 

8 

7 

3.2 

1 

4 

3 

5 

6  2 

8 

7 

4.2 

1 

4 

3 

5 

6  2 

8 

7 

5.2 

1 

4 

3 

5 

6  2 

8 

7 

6.1 

1 

4 

3 

5 

6  2 

8 

7 

6.2 

1 

4 

3 

5 

6  2 

8 

7 

Devices 

M 

P 

M 

S 

C  P 

C 

S 

Note.  Letters 

refer  to 

devices 

• 

« 

c  = 

P-COFT;  M  =  MILES; 

P 

=  PRIME; 

S  =  Subcaliber. 

Numbers 

denote  tasks  within  clusters: 

1  = 

travel ; 

2  =  drills;  3  =  bound;  4  =  overwatch;  5  =  occupy;  6  =  maneuver; 
7  =  patterns;  8  =  techniques. 


The  next  problem  was  to  resolve  the  sequence  of  subtasks,  at  level  D  of 
the  hierarchy,  within  the  tasks  ordered  above.  Tasks  3  (Bound),  4 
(Overwatch),  and  6  (Maneuver)  each  consisted  of  only  one  subtask.  Hence,  it 
was  only  necessary  to  seauence  within  the  remaining  tasks  1  (Travel),  2 
(Drills),  5  (Occupy),  7  (Fire  patterns),  and  8  (Firing  techniques).  These 
were  considered  in  the  order  chosen  above  in  Table  11  (1,  5,  2,  8,  7). 

Applying  sequencing  principles  to  these  series  of  subtasks  gave  rise  to  the 
results  listed  in  Table  12.  Successive  modifications  are  not  shown  in  detail, 
since  the  applicable  principles  appeared  to  generate  similar  orders. 

The  final  step  is  to  complete  the  assignment  of  device  support  to 
subtasks.  There  was  sufficient  variety  at  the  subtask  leve:  to  justify  making 
direct  use  of  the  utility  ratings.  Table  13  summarizes  the  final  order  of 
subtasks,  associating  each  with  the  devices  ranked  first  and  second  in 
utility.  The  tabulation  demonstrates  that  there  was  ample  scope  for  providing 
training  on  a  variety  of  supporting  devices. 
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Table  12 


Final  Orders  for  Subtasks  Within  All  Platoon  Collective  Tasks 


Task 

Subtask  Order 

1.  Travel 

1.5 

Column 

1.3  1.2  1.4 

Line  Echelon  Vee 

1.1 

Wedge 

4.  Overwatch 

4.0 

Overwatch 

3.  Bound 

3.0 

Bound 

5.  Occupy 

5.1 

Initial 

5.2 

Subsequent 

6.  Maneuver 

6.0 

Maneuver 

2.  Drills 

2.1 

Action 

2.2 

Contact 

2.3 

Air 

8.  Techniques 

8.1 

Observed 

8.3 

Simult 

8.2 

Altern 

7.  Patterns 

7.1 

Frontal 

7.3 

Depth 

7.2 

Cross 

The  final  ordering  summarized  in  Table  13  embodies  virtually  all  of  the 
sequencing  recommendations  discussed  above.  In  so  far  as  the  principles  are 
applicable,  the  sequence  follows  the  logic  of  the  subject  matter  (principle 
1.2)  and  uses  progressive  elaboration  (1.3).  Wherever  possible,  the  training 
moves  from  slow  to  faster  practice  (3.1).  In  keeping  with  most  training 
practices,  the  sequence  progresses  from  easy  to  difficult  tasks  (4.2). 

Training  in  the  recommended  order  follows  the  natural  sequence  of  the  platoon 
functions  (5.2),  thus  concurrently  preserving  job  performance  order  (2.1), 
chronological  sequence  (2.2),  and  critical  sequences  (2.3).  The  recommended 
training  sequence  confers  an  appreciable  amount  of  variety  of  practice  (6.1), 
thus  facilitating  the  generalization  necessary  for  efficient  transfer  of 
training.  Finally,  as  evidenced  by  Table  13,  the  training  schedule  will  offer 
opportunities  for  practice  on  the  real  equipment  early  in  the  training 
sequence  (6.2),  thus  providing  for  the  establishment  of  appropriate  cognitive 
models  and  correcting  for  failures  in  device  fidelity. 
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Table  13 

Training  Recommendations  Using  Ordered  Subtasks  with  Assigned  Device  Support 


Subtask 

Trainina 

Device 

Order 

Number 

T  itle 

Primary 

Secondary 

1 

1.5 

Execute  a  Column/Staggered  Column 

Subcaliber 

P-COFT 

2 

1.3 

Execute  a  Line  Formation 

MILES 

P-COFT 

3 

1.2 

Execute  an  Echelon  Formation 

Subcaliber 

P-COFT 

4 

1.4 

Execute  a  Vee  Formation 

P-COFT 

MILES 

5 

1.1 

Execute  a  Wedge  Formation 

Subcaliber 

P-COFT 

6 

4.0 

Overwatch  a  Bounding  Platoon 

P-COFT 

MILES 

7 

3.0 

Bound  by  Section 

MILES 

PRIME 

8 

5.1 

Occupy  Initial  Battle  Position 

MILES 

P-COFT 

9 

5.2 

Occupy  Subsequent  Battle  Position 

Subcaliber 

P-COFT 

10 

6.0 

Maneuver  Within  a  Battle  Position 

Subcal iber 

P-COFT 

11 

2.1 

Execute  Action  Drill 

MILES 

PRIME 

12 

2.2 

Execute  Contact  Drill 

P-COFT 

MILES 

13 

2.3 

Execute  Air  Attack  Drill 

Subcaliber 

P-COFT 

14 

8.1 

Employ  Observed  Fires 

P-COFT 

Subcaliber 

15 

8.3 

Employ  Simultaneous  Fires 

P-COFT 

MILES 

16 

8.2 

Employ  Alternating  Fires 

P-COFT 

Subcaliber 

17 

7.1 

Employ  Frontal  Fire 

P-COFT 

Subcaliber 

18 

7.3 

Employ  Depth  Fire 

P-COFT 

Subcaliber 

19 

7.2 

Employ  Cross  Fire 

P-COFT 

Subcaliber 

Summary  and  Conclusions 

The  application  of  methods  for  designing  a  training  strategy 
demonstrated,  in  general,  that  the  various  methods  could  be  applied  to 
dissimilar  gunnery  training  problems  with  sensible  results.  Only  one  of  the 
methods  failed  to  apply  to  both  problems:  The  methods  for  allocation  of 
training  time  were  only  applicable  to  the  simpler  gunnery  training  problem 
where  only  two  training  devices  were  considered;  the  methods  for  allocating 
training  time  were  not  well-suited  for  the  more  difficult  multi-task,  multi¬ 
device  tactical  training  problem.  The  problem  was  not  in  the  mathematics: 

The  mathematical  methods  for  allocating  training  time  described  by  Cronholm 
(1985)  and  Sticha,  Singer  et  al.  (1988)  could  have  been  extended  to  the  multi¬ 
task,  multi-device  allocation  problem.  Rather,  the  problem  was  that  data 
requirements  of  such  methods  exceeded  that  which  could  be  reasonably  obtained. 
Clearly,  no  performance  data  exist  for  this  complex  problem,  leaving  simulated 
transfer  data  as  the  only  alternative.  To  obtain  simulated  transfer  data, 
experts  would  have  been  required  to  make  numerous  demanding  judgments  about 
the  capabilities  of  training  devices,  some  of  which  exist  only  in  prototype 
form,  to  train  subtasks  for  which  there  are  no  clear-cut  standards  of 
performance.  The  resulting  data  collected  under  those  conditions  would  be 
virtually  meaningless  and  incongruent  with  their  treatment  by  sophisticated 
mathematical  techniques. 


As  reflected  in  the  detailed  results  from  this  applications  section  of 
the  report,  differences  between  the  problems  were  noted  in  the  level  of  effort 


required  to  apply  the  methods.  For  instance,  it  was  considerably  more 
difficult  to  apply  the  instructional  sequencing  methods  to  the  second  tactical 
problem  than  to  the  first  problem  involving  basic  gunnery  skills.  The  reason 
for  this  difference  was  simply  the  greater  number  of  objectives  in  the  second 
problem  which  led  to  a  much  greater  number  of  possible  sequences. 

Consequently,  the  sequencing  rules  were  applied  in  a  more  systematic  manner  in 
the  second  compared  to  the  first  problem  to  insure  overall  adherence  to  the 
rules.  On  the  other  hand,  the  instructional  structuring  methods  were  more 
complex  for  the  first  than  for  the  second  problem.  The  cause  of  this 
difference  appeared  to  be  the  fact  that  the  first  problem  dealt  with  "smaller" 
objectives  that  were  more  likely  to  be  related  to  one  another  than  the 
tactical  objectives  in  the  second  problem.  Analogous  to  the  previous  case, 
the  approach  taken  was  to  be  systematic  and  consider  all  relations  among 
objectives. 

The  fact  that  there  were  differences  between  problems  in  exact 
methodology  argues  that  fully  developed  algorithms  cannot,  at  the  present 
time,  be  developed  for  the  instructional  sequencing  and  structuring 
strategies.  The  appendix  addresses  this  problem  indirectly  by  suggesting 
systematic  procedures  for  using  the  heuristics  related  to  structuring  and 
sequencing  instruction.  However,  there  may  be  some  benefit  in  the  methods 
remaining  somewhat  undetermined  so  that  the  user  can  tailor  them  to  his  or  her 
particular  training  application.  On  the  other  hand,  without  algorithmic 
procedures,  there  is  no  guarantee  that  different  users  will  arrive  at 
identical  training  strategies.  However,  if  different  strategies  were  obtained 
from  appropriate  application  of  the  methods,  the  users  would  be  assured  that 
both  would  be  in  accord  with  basic  training  principles.  In  this  regard,  the 
outcome  from  the  application  of  a  heuristic  can  be  seen  to  be  different  than 
that  from  an  algorithm:  Heuristic  methods  do  not  necessarily  provide  the  user 
with  a  single  "optimal"  strategy;  rather,  they  provide  basic  guidance  for 
developing  a  strategy  that  adheres  to  known  training  principles. 

Although  the  training  strategy  methods  were  discussed  and  applied 
independently,  there  are  many  points  at  which  they  interact.  For  instance, 
organizing  objectives  affects  the  sequencing  of  those  objectives  and  vice 
versa.  Reigeluth  and  Curtis  (1987)  argued  that,  because  of  these 
interactions,  "...  strategy  components  should  not  be  prescribed 
individually;  instead,  they  should  be  combined  into  optimal  models  .  .  .  for 
different  learning  situations"  (p.  177).  Such  models  are  starting  to  appear. 
For  instance,  the  model  for  Optimization  of  Simulation-Based  Training  Systems 
(OSBATS)  is  concerned  with  the  training  device  design  and  use  (Sticha, 
Blacksten,  et  al.,  1988).  The  present  concern,  in  contrast,  is  with  the  use 
of  existing  devices  for  training  and  testing  purposes.  More  in  line  with  the 
present  purposes,  Blacksten  (1989)  provided  the  outline  of  an  integrated  model 
of  media  selection  that  is  based  on  learning  and  transfer  functions.  The 
outcome  not  only  selects  the  media,  but  allocates  training  time  as  well.  Both 
approaches  are  mathematically  complex  and  have  substantial  data  requirements. 
Furthermore,  it  is  arguable  whether  or  not  the  current  knowledge  of  training 
processes  can  support  such  sophisticated  optimal  models.  The  present 
approach,  which  summarizes  this  knowledge  in  heuristic  and  algorithmic 
guidelines,  is  perhaps  an  intermediate  point  in  developing  such  an  integrated 
model.  As  these  components  are  applied  and  empirically  tested,  an  integrated 
model  for  developing  training  strategies  may  eventually  emerge. 
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Appendix 

Methods  and  Materials  for  Structuring  and  Sequencing  Training 

As  discussed  in  the  main  body  of  the  report,  the  methods  for  selecting 
devices  and  allocating  training  are  relatively  well-developed  algorithms.  In 
contrast,  the  methods  for  structuring  and  sequencing  training  are  presented  as 
a  set  of  heuristic  guidelines.  Some  of  the  guidelines  were  based  on  well- 
established  research  findings  and  training  practices,  while  others  were  based 
on  some  recent  theorizing.  Despite  the  tentative  nature  of  the  guidelines,  it 
is  still  possible  to  specify  how  they  might  actually  be  used;  the  purpose  of 
this  appendix  is  to  do  just  that.  The  present  methods  are  presented  in  this 
appendix  instead  of  the  body  of  the  report,  because  they  are  unprecedented  and 
untried.  Nevertheless,  this  initial  scheme  provides  the  bare-bones  outline  of 
a  method  from  which  more  algorithmic  procedures  may  be  developed.  Development 
of  the  present  procedures  are  discussed  below  in  terms  of  three  related 
issues:  (a)  the  information  requirements  of  the  methods,  (b)  the  format  of 
the  input  data,  and  (c)  the  use  of  the  output  data. 

Information  Requirements 

As  discussed  in  the  text,  the  guidelines  for  structuring  and  sequencing 
training  require  detailed  knowledge  about  training  concepts  and  methods.  It 
is  assumed  that  the  user  of  the  present  methods  is  an  experienced  training 
designer  who  would  him/herself  play  the  role  of  training  methods  expert.  The 
other  requirement  is  detailed  knowledge  of  the  task  domain  being  trained,  that 
is,  armor  gunnery.  The  topics  of  gunnery  and  gunnery  training  have  become 
increasingly  technical  and  specialized  in  recent  years.  As  a  result,  it  is 
unreasonable  to  expect  the  training  designer  to  have  the  specialized  knowledge 
to  execute  the  methods.  In  that  case,  the  designer  must  rely  on  input  from 
subject  matter  experts  (SMEs).  One  specific  purpose  of  the  present  appendix 
is  to  specify  more  clearly  the  required  input  of  SMEs. 

The  process  of  specifying  SME  input  began  with  the  identification  of 
those  guidelines  where  their  input  would  be  most  helpful.  Table  14  summarizes 
the  heuristic  guidelines  that  were  presented  in  the  text  of  the  report,  and 
indicates  whether  or  not  useful  input  can  be  obtained  from  a  training  methods 
expert  or  from  an  SME.  As  shown  in  the  table,  both  types  of  expertise  are 
appropriate  to  most  of  the  guidelines.  The  only  cases  where  SMEs  do  not  would 
not  have  an  obvious  role  in  providing  information  are  those  guidelines  that 
relate  to  the  specification  of  training  conditions.  Also,  some  of  these  rules 
come  into  play  later  in  the  training  design  process,  that  is,  after  training 
devices  have  been  chosen.  Nevertheless,  this  initial  analysis  suggests  that 
the  SME  can  provide  significant  input  to  the  application  of  these  guidelines. 

Format  for  SME  Input 

The  next  issue  is  what  format  the  SME  input  should  take.  One  approach 
used  in  the  training  development  literature  is  to  have  experts  provide 
numerical  ratings  of  the  elements  of  the  instruction.  For  instance,  SMEs 
typically  provide  numerical  ratings  of  the  criticality  of  training  objects; 
these  ratings  are  the  used  to  arrange  training  priorities  within  a  course  of 
instruction.  This  quantitative  approach  allows  the  training  designer  to 
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Table  A-l 

Sources  of  Information  for  Structuring  and  Sequencing  Principles 


Source  of 

Information 

Trainina  Strateav  Problem 

Guidelines 

Training 

Methods 

Expert 

Subject 

Matter 

Expert 

Structure  Trainina  Objectives 

A  pair  of  objectives  should  be  trained  together 
given  that  one  of  the  following  conditions  is  met: 

•  one  objective  is  dependent  on  the  other  (i.e.,  it 
has  the  other  as  a  prerequisite); 

YES 

YES 

•  one  objective  supports  the  other  (i.e.,  it 
transfers  to  the  other); 

YES 

YES 

•  both  objectives  are  sequentially  related  and 
share  a  common  task  subgoal;  or 

YES 

YES 

•  both  objectives  are  time-shared  and  share  a 
common  attentional  resource. 

YES 

YES 

Sequence  Trainina 

Sequence  training  according  to  the  following 
guidelines: 

•  Teach  ends  before  means 

YES 

NO 

•  Follow  the  logic  of  the  subject  matter 

NO 

YES 

•  Use  progressive  elaboration 

YES 

YES 

•  Move  from  slow  to  fast  practice 

YES 

NO 

•  Move  from  orientation,  through  practice, 
to  real  task 

YES 

NO 

•  In  general,  progress  from  easy  to  difficult 

YES 

YES 

•  Follow  the  natural  sequence  of  the  task 

NO 

NO 

•  Vary  practice  between  training  devices 

YES 

NO 

•  Provide  real  equipment  experience  early  in 
training 

YES 

NO 
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easily  aggregate  the  input  from  multiple  SMEs  by  calculating  measures  of 
central  tendency  (e.g.,  the  mean,  median,  or  mode).  These  quantitative 
ratings  also  allow  the  designer  to  measure  the  agreement  among  the  various 
raters.  In  that  regard,  it  should  be  noted  that  interrater  reliability  is  a 
necessary  (but  not  sufficient)  condition  for  valid  ratings.  One  problem  with 
this  approach  is  that  people  are  not  typically  very  reliable  in  their 
subjective  ratings  of  individual  objects.  For  instance,  interrater 
reliability  for  rating  the  criticality  of  individual  tasks  is  poor;  however, 
reliability  substantially  improves  when  they  are  asked  to  make  relative 
judgments  of  pairs  of  objects  (Harris,  Osborn,  &  Boldovici,  1977). 

To  the  extent  possible,  this  paired  comparison  format  was  used  in  the 
proposed  methods.  That  is,  paired  comparison  items  were  developed  to 
correspond  to  those  guidelines  wherein  SME  input  was  identified  as  relevant. 
For  some  of  those  guidelines,  however,  a  paired  comparison  format  was  deemed 
inappropriate;  for  those,  absolute  ratings  were  developed.  The  attachment  to 
this  appendix  presents  both  the  paired  comparison  and  individually  rated  items 
in  the  form  of  a  worksheet  that  SMEs  can  use  to  provide  their  input. 

With  respect  to  the  paired  comparisons,  two  points  should  be  noted. 

First,  there  is  a  potential  problem  in  the  number  of  judgments  required  by  the 
paired  comparisons  approach.  For  k  objects,  there  are  k(k  -  l)/2  possible 
comparisons.  That  means  that,  for  a  list  of  100  objectives  for  instance, 
there  would  be  4,950  pairs  that  must  be  rated.  In  those  cases  where  the 
numbers  of  to-be-rated  pairs  is  unreasonable  for  an  individual  judge  to 
manage,  the  partial  paired  comparison  method  can  be  used  wherein  individual 
raters  provide  judgments  on  selected  subsets  of  the  total  numbers  of  to-be¬ 
rated  pairs,  McCormick  and  Bachus  (1952)  showed  that  a  partial  paired 
comparison  approach  can  be  used  without  serious  degradation  in  reliability. 
Second,  some  items  are  not  "pure"  paired  comparison  in  the  sense  that  the 
rater  is  not  presented  a  forced  choice  of  one  object  over  the  other. 

Sometimes  the  ratings  are  on  some  continuous  scale.  For  example,  two  of  the 
items  require  the  rater  to  judge  the  amount  of  transfer  that  would  occur  from 
one  objective  to  the  other  in  the  pair.  Even  though  such  items  do  not  use  a 
forced  choice  format,  the  paired  comparison  format  is  useful  in  isolating  the 
pairwise  relationships  that  exist  among  the  objectives. 

The  worksheet  items  are  clustered  into  groups  that  relate  to  specific 
concepts  discussed  in  the  text  of  the  report.  These  concepts  are  explained  to 
the  raters  using  examples  with  which  they  should  be  familiar.  If  the  SME  were 
to  rate  many  objectives  or  objective  pairs,  it  might  be  advantageous  to  devise 
separate  instructions  and  response  sheets.  These  groups  of  conceptually 
related  items  are  explained  below,  and  their  implications  for  training  are 
briefly  noted. 

1.  Prereguisiteness.  Items  la  and  lb  concern  the  concept  of 
prerequisiteness.  Items  related  by  prerequisiteness  should  be  trained  in 
close  proximity  to  minimize  forgetting  and  interference  effects.  Also  the 
prerequisite  must  be  trained  prior  to  superordinate  objectives. 

2.  Transfer  among  objectives.  Objectives  that  are  related  by  transfer 
should  be  kept  close  together  to  minimize  forgetting  and  interference  effects. 
As  discussed  in  the  text,  however,  transfer  relationships  do  not  have  clear 
implications  for  the  sequencing  of  training.  Items  2a  and  2b  require  the  SME 
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to  predict  the  transfer,  in  terms  of  time  savings,  that  would  occur  from  one 
member  of  the  pair  to  the  other. 

3.  Interference  of  simultaneous  activities.  Items  3a  -  3c  pertain  only 
to  individual  crewman's  activities  that  can  be  performed  together.  If  the 
pair  of  activities  interfere  with  one  another,  they  should  be  trained  together 
so  that  the  performer  learns  appropriate  time-sharing  skills.  If  they  are  not 
interfering,  they  could  be  trained  apart  (if  need  be).  [This  concept  of 
interfering  simultaneous  activities  is  also  addressed  in  7  below,  but  in  a 
more  theoretical  context.] 

4.  Job  performance  order.  The  purpose  of  Item  4  is  to  ascertain  the 
sequential  relationships  that  exist  between  activities  corresponding  to  a  pair 
of  training  objectives.  The  implication  is  that,  if  such  sequential 
relationships  exist  on  the  job,  they  ought  to  be  maintained  in  training. 

5.  Ease  of  learninq/performance.  Items  5a  and  5b  pertain  to  the 
relative  ease  of  the  two  objectives.  The  first  item  of  this  group  pertains  to 
the  ease  of  learning  the  objective,  whereas  the  second  relates  to  the  ease  of 
performing  the  activities  that  correspond  to  the  objective.  All  other  factors 
being  equal,  training  on  an  easier  objective  should  precede  the  training  of  a 
rr.ore  difficult  one. 

6.  Common  subqoals.  The  first  of  the  individual  ratings  pertains  to 
the  guidance  that  objectives  related  to  a  common  subgoal  should  be  trained 
together.  If  the  domain  were  not  already  divided  into  subgoals,  one  possible 
approach  is  to  have  SMEs  sort  objectives  into  piles--the  Q-sort  technique--to 
discover  any  implicit  goal  structure  in  the  domain  (Glaser  et  a  1.,  1986). 
Morrison  (1984)  presented  a  method  for  analyzing  the  proximity  of  recalled 
elements  for  revealing  the  implicit  goal  structure  of  procedural  tasks. 
However,  it  is  more  likely  that  as  that  the  training  objectives  nave  been 
derived  through  an  analysis  of  domain  subgoals  (Morrison,  et  a  1 . ,  1990).  If 
that  be  true,  the  more  expeditious  approach  would  have  SMEs  confirm  this 
explicit  structure  by  identifying  which  objectives  are  associated  with  the 
subgoals  that  have  been  identified. 

7.  Attentional  resources.  Item  7  is  designed  to  supplement  Items  3a-3c 
by  describing  the  objective  with  respect  to  Wickens's  (1989)  "resource  space." 
As  discussed  in  the  text,  this  space  is  used  to  predict  the  extent  to  which 
two  simultaneously  performed  activities  can  be  time-shared.  The  items  pertain 
to  the  three  dimensions  of  the  resource  space:  stage  of  processing,  modality 
of  processing,  and  type  of  response.  The  general  prediction  from  this  model 
is  that  two  activities  that  require  common  levels  on  a  dimension  will  suffer 
greater  interference  than  two  that  demand  separate  levels. 

Use  and  Integration  of  5ME  Information 

As  stated  earlier,  the  training  methods  expert  may  want  *o  make 
him/herself  available  during  the  SME  input  to  answer  questions  that  may  come 
up--particularly  with  regard  to  some  of  the  more  abstract  concepts  discussed 
above.  In  that  regard,  a  workshop  of  SMEs  and  the  training  methods  expert  may 
facilitate  the  interchange  between  the  two  types  of  experts,  and  aid  the 
training  expert  in  providing  his/her  input. 


Once  the  input  is  obtained,  the  next  question  is  how  to  apply  the 
guidance.  The  guidelines  were  not  designed  to  be  applied  in  any  particular 
order;  rather,  training  designers  should  use  the  information  from  the  SMEs  and 
their  own  knowledge  to  construct  tentative  configurations  to  fit  those 
guidelines.  Morrison  (1985)  suggested  using  index  cards  to  represent  each 
objective  and  trying  to  arrange  the  course's  configuration  so  that  it  is 
maximally  congruent  with  guidelines.  This  process  is  viewed  as  an  iterative 
procedure  wherein  the  designer  successively  resequences  and  restructures  the 
course  to  achieve  better  and  better  congruency  with  the  whole  set  of 
guidel ines. 

Because  these  methods  are  new  and  untested,  they  should  be  the  focus  for 
continuing  research  and  development.  One  area  of  development  might  be  to 
treat  the  SME  data  to  more  sophisticated  analyses  than  described  above.  For 
instance,  the  degree  to  which  pairs  of  objectives  are  interfering  can  be  used 
as  proximity  data  in  a  multidimensional  analysis  of  task  components.  The 
result  would  provide  an  empirically  derived  structural  model  of  attentional 
resources.  It  would  be  of  both  theoretical  and  practical  interest  to  examine 
the  degree  to  which  this  solution  corresponds  to  Wickens's  (1989)  resource 
space. 


Attachment  to  the  Appendix 
Worksheet  for  SME  Ratings 


Paired  Comparisons 


The  following  questions  concern  the  following  two  training  objectives: 


Objective  A: 

Title . 

Objective  B: 

Title . 

Prerequisiteness 

The  first  rating  issue  relates  to  the  necessity  of  learning  one 
objective  before  starting  to  learn  the  other.  For  instance,  we  must  learn  to 
walk  before  we  can  learn  to  run.  In  terms  of  academic  skills,  we  must  learn 
to  count  before  we  can  learn  addition. 


QUESTION 

RESPONSE 

la.  Is  it  necessary  to  know  A 
before  B  can  be  learned? 

YES  NO  (Circle  one) 

lb.  Is  it  necessary  to  know  B 
before  A  can  be  learned? 

YES  NO  (Circle  one) 

Transfer 


Although  it  may  not  be  necessary  to  learn  one  objective  before  another, 
having  learned  one  objective  may  nevertheless  be  helpful  to  learning  another. 
By  "helpful,''  we  mean  that  having  learned  one  objective  will  actually  speed  up 
learning  on  the  second.  For  instance,  it  is  not  necessary  to  learn  to  drive  a 
car  before  learning  to  drive  a  truck;  however,  a  person  who  already  knows  how 
to  drive  a  car  will  learn  truck  driving  more  quickly  than  a  person  who  has 
never  driven  a  car. 

The  following  ratings  are  in  terms  of  the  percent  of  time  required:  If 
X  does  not  help  at  all,  enter  0;  if  X  reduces  the  time  required  by  half,  enter 
50;  if  X  reduces  the  time  required  by  three-quarter,  enter  75;  and  so  on. 


QUESTION 

RESPONSE 

2a.  Rate  the  percent  of  time  saved 

(0-100  percent) 

learning  B  after  having  learned 

A  to  standard. 

2b.  Rate  the  percent  of  time  saved 

(0-100  percent) 

learning  A  after  having  learned 

B  to  standard. 
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Interference  of  Simultaneous  Activities 

Seme  tasks  can  sometimes  be  performed  simultaneously.  For  instance,  a 
pilot  often  must  use  the  radio  while  flying  his  aircraft.  The  following 
questions  require  you  to  indicate  whether  or  not  the  activities  described  by  A 
and  B  can  be  performed  simultaneously. 


QUESTION 

RESPONSE 

3a.  Can  the  activities 

corresponding  to  A  and  B  be 
performed  simultaneously? 

YES  (Answer  3b-3d) 

NO  (Go  to  4) 

If  you  answered  YES  to  3a,  rate  the  degree  to  which  performance  one  activity 
degrades  performance  on  the  other.  If  one  activity  completely  disrupts  the 
other,  enter  100%;  if  one  activity  degrades  performance  by  half,  enter  50%;  if 
one  activity  has  no  effect  on  the  performance  on  the  other,  enter  0%. 


QUESTION 

RESPONSE 

3b.  Is  it  possible  that  A  and  B  can 
be  performed  at  the  same  time? 

YES  NO  (Circle  one) 

3c.  If  response  to  3a  is  YES,  rate 
the  degree  to  which  performance 
of  A  degrades  performance  of  B. 

(0-100  percent ) 

3d.  If  response  to  3a  is  YES,  rate 
the  degree  to  which  performance 
of  B  degrades  performance  of  A. 

(0-100  percent) 
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Sequential  Relationships 

On  the  job,  the  activities  related  to  one  objective  may  precede  the 
activities  of  another.  For  example,  learning  to  change  a  tire  might  could  be 
decomposed  into  two  training  objectives:  using  the  jack  to  raise  the  car,  and 
removing  the  wheel  and  tire.  Clearly,  the  former  objective  usually  precedes 
the  second.  Another  possibility  is  that  the  two  objectives  may  not  be 
sequentially  related.  For  instance,  there  is  no  necessary  sequential 
relationship  between  learning  to  change  a  tire  and  learning  to  shift  gears. 


QUESTION 

RESPONSE 

4.  Describe  the  sequential 
relationship  that  exists 
between  the  activities  in 
objectives  A  and  B. 

_  A  normally  occurs  before  B 

_  B  normally  occurs  before  A 

_  A  and  B  are  not  sequentially 

related  (Check  one) 

Ease  of  Learning  and  Performance 

The  next  group  of  items  concern  the  relative  ease  of  the  two  objectives. 
Item  5a  asks  you  to  identify  which  of  the  two  objectives  is  easier  to  learn, 
whereas  item  5b  asks  which  is  easier  to  perform. 


QUESTION 

RESPONSE 

5a.  Of  the  two  objectives,  which  is 

_  Objective  A 

the  easier  to  learn? 

_  Objective  B 

(Check  one) 

5b.  Once  learned,  which  of  the  two 

_  Objective  A 

is  easier  to  perform? 

_  Objective  B 

(Check  one) 
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Individual  Ratings 


Objective  A:  T-«tle. 


Subqoals 

Tasks  are  performed  for  a  reason.  These  reasons  are  called  task  goals 
and  subgoals.  For  instance,  raising  a  automobile  with  a  jack  could  be 
classified  as  a  task.  However,  it  is  performed  with  some  specific  goal  in 
mind  (e.g.,  to  change  a  tire).  Similarly,  we  have  identified  the  general 
subgoals  to  which  the  activities  corresponding  to  objectives  could  be 
addressed. 


QUESTION 


6.  Identify  which  of  the  subgoals 
Objective  A  addresses. 


RESPONSE 


_  Subgoal  1 

_  Subgoal  2 

_  Subgoal  3 

_____  Subgoal  4 


Resource  Space 

Tasks  can  be  described  with  respect  to  their  mental  demands.  The 
demands  can  be  stated  in  terms  of  three  dimensions:  stage  of  processing, 
sensory  modality,  and  type  of  responses.  For  instance,  "issue  fire  command" 
would  be  classified  as  involving  primarily  central  processing,  using  both 
visual  and  auditory  modalities,  and  requiring  a  vocal  (as  opposed  to  manual) 
response.  Using  the  response  alternatives  provided  below,  describe  Objective 
A  with  respect  to  these  three  dimensions. 


QUESTION 

RESPONSE 

7a.  Describe  the  objective  with 

encoding  (i.e.,  perception, 

respect  to  "stage  of 

recognition) 

processing" 

_  central  processing  (i.e., 

memory,  cognition) 

_  overt  responding 

7b.  Describe  the  objective  with 

_  vision 

respect  to  sensory  modality 

_  audition 

7c.  Describe  the  objective  with 

manual 

respect  to  type  of  responses 

_  vocal 

